User-agent: * Disallow: /cgi-bin/ Disallow: /tmp/ Disallow: /cache/ Disallow: /class/ Disallow: /include/ Disallow: /install/ Disallow: /kernel/ Disallow: /SEMrushBot/ User-agent: SemrushBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: MauiBot Disallow: / User-agent: PetalBot Disallow: / User-agent: MJ12Bot Disallow: / User-agent: AspiegelBot Disallow: / User-agent: DotBot Disallow: /