Depending on how IPV6_V6ONLY is set (default is /proc/sys/net/ipv6/bindv6only and can be changed per application), the above might only be listening for IPv6 connections.
So if you are trying to connect to IPv4 you might be connecting to something else.
Check with "ss -an" or "netstat -n --tcp -a" and see if there is a separate socket listening for IPv4 connections.
If there is only one listener, it is IPv6, and it accepts IPv4 connections, then I am on the wrong track entirely.