User-agent: * Disallow: /api/ Disallow: /my/ Disallow: /me/ Disallow: /email/ Disallow: /d/ User-agent: Twitterbot Allow: /me/referee/registerko/ Allow: /m/