Skip to main content ↑ ↓
Topic: What should we mark as noindex (Read 2387 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

What should we mark as noindex

This topic has come up once or twice, regarding what we should mark as noindex by default. 

Several of our templates make use of this, but due to the way some things work with layers etc, the place to set the noindex directive is back in the controllers.  This is fine, although not necessary convenient from an end users looking to change the default behavior.

Using the robot.txt file to block access to certain areas works, however it does NOT prevent google from showing those assets in search results should google find a link to that asset. 
QuoteImportant! For the noindex meta tag to be effective, the page must not be blocked by a robots.txt file. If the page is blocked by a robots.txt file, the crawler will never see the noindex tag, and the page can still appear in search results, for example if other pages link to it.

So I was thinking we could add a $context['noindex'] array, probably in the loadTheme jumbo, that sets noindex for many of the areas that don't really have a reason to be indexed .... as an example
Code: [Select]
	$context['noindex'] = array('profile', 'search', 'calendar', 'memberlist', 'help',  'who',  'stats', 'login', 'reminder', 'register',  'verificationcode', 'contact');

Then in index.template update the existing line to be
Code: [Select]
	// Please don't index these Mr Robot.
	if (!empty($context['robot_no_index']) || (!empty($context['current_action']) && in_array($context['current_action'], $context['noindex'])))
		echo '
	<meta name="robots" content="noindex" />';

Thoughts?

Re: What should we mark as noindex

Reply #1 –

 emanuele likes! [1]

Since in 1.1 we have a Theme class (actually two lol), it may be an option to have the array as attribute of the class and then have in $context just "no_index" = true/false
Something like:
Code: (sources/Theme.php) [Select]
protected $no_index_actions = array();

public function __construct()
{
...
$this->no_index_actions = array('profile', 'search', 'calendar', 'memberlist', 'help',  'who',  'stats', 'login', 'reminder', 'register',  'verificationcode', 'contact');
...
}

then:
Code: (themes/default/Theme.php) [Select]
public function setupThemeContext()
{
...
$context['robot_no_index'] = in_array($context['current_action'], $this->no_index_actions);
...
}

and finally in the template:
Code: [Select]
	// Please don't index these Mr Robot.
	if (!empty($context['robot_no_index']))
		echo '
	<meta name="robots" content="noindex" />';

This would still allow the theme to define additional no_indexes like:
Code: (themes/my_theme/Theme.php) [Select]
public function __construct()
{
parent::__construct();
$this->no_index_actions[] = 'my_action';
}

Well, not a lot of advantages in the end, just one less index in context. xD

BTW I like the idea. nods
1 And I think it would be wonderfully cool to have a trickery that reads the me likes and converts it into a like to the OP post. lol :P↵
Bugs creator.
Features destroyer.
Template killer.

Re: What should we mark as noindex

Reply #2 –

Done ... on my local ... PR in a bit, moar stuff to add  ;)

Re: What should we mark as noindex

Reply #3 –

Are all those things really not interesting? Okay, I can definitely see why you wouldn't want profile stuff not randomly showing up, but if you search for something like "site:elkarte.net register" then I think it would be straightstrange if you didn't get any (particularly relevant) results. That is, I'd be more inclined to think in terms of a reason to not be indexed than having to come up with a reason why something should be indexed.

Edit: fixed an autocorrect typo.
Last Edit: May 20, 2017, 12:18:22 pm by Frenzie

Re: What should we mark as noindex

Reply #4 –

The access to the various sections is generally regulated by permissions, but if a link is present (e.g. for profiles) and guests do not have permissions, the search engine will collect a series of pointless login forms.
The noindex would clean them up a bit at least. :)
Bugs creator.
Features destroyer.
Template killer.

Re: What should we mark as noindex

Reply #5 –

Right, a lack of permissions for guests would be a reason for noindex.