Ques/Help Recommended robots.txt for XF 2.3

[frm]

I found this, and used a modified version of it for XF 2.2.

However, I was wondering if any changes should be made for XF 2.3

Old Nick said:
Here is mine
Code:

User-agent: PetalBot
User-agent: AspiegelBot
User-agent: AhrefsBot
User-agent: SemrushBot
User-agent: SemRush
User-agent: DotBot
User-agent: MauiBot
User-agent: MJ12bot
Disallow: /

User-agent: Amazonbot
Disallow: /threads/*/reply

User-agent: *
Disallow: /whats-new/
Disallow: /account/
Disallow: /members/
Disallow: /attachments/
Disallow...

Click to expand...

Read more

Continue reading...