Getting Unauthorised 401 for my nextjs 13 app on Search Console

Question

I built out a nextjs 13 webapp using clerk for auth. The issue is that the homepage of the web app isn't discoverable on Google. when i Test Live URL, this is how it looks: enter image description here

I've done basic implementation of seo in the app. But i see this in vercel whenever i try to crawl the website: enter image description here

I suspected Clerk Middleware is the issue, but i've added homepage in public routes enter image description here

If someone can help, that'd be very helpful!

Answer 1

I was getting Unauthorized using Clerk middleware as well, but I couldn't figure out a solution, so I just went back to the previous middleware version and everything worked fine.

import { withClerkMiddleware } from "@clerk/nextjs/server";
import { NextResponse } from "next/server";
import type { NextRequest } from 'next/server'

export default withClerkMiddleware((req: NextRequest) => {
  return NextResponse.next();
});

// Stop Middleware running on static files
export const config = { matcher:  '/((?!_next/image|_next/static|favicon.ico).*)',};

Answer 2

Check if you are using Clerk's Middleware. If that's the case then this error is caused by the fact that you are on the development version instead of production.