It will only issue wildcards if you have any sites named like *.yourdomain.com, i.e. it needs to see the *. to know to issue wildcards.

The relevant parts of my Caddyfile look like this:

{
	# TLS settings.
	acme_dns cloudflare {env.CLOUDFLARE_API_TOKEN}
	email {env.ACME_EMAIL}
}

# Proxy a subdomain to a backend server.
# Usage: `import proxy subdomain backendHost`
(proxy) {
	@sub-{args.0} host {args.0}.{$DOMAIN}
	handle @sub-{args.0} {
		reverse_proxy http://{args.1}
	}
}

# Put everything in the same block to get a wildcard certificate.
*.{$DOMAIN} {
	# Handle particular subdomains.
	import proxy changedetection changedetection:5000
	import proxy uptime uptime-kuma:3001
	import proxy whoami whoami

	# Fallback message (unknown subdomain).
	handle {
		error "This subdomain is not currently in use." 404
	}
}

The (alias) snippet at the top is used in the site block to tell it how to use a particular subdomain.

(I've removed some Authelia stuff and handling the apex domain)

{$DOMAIN} fills in my base domain from the environment, and {env.*} does the same for my credentials (but without putting it in the JSON config).