Can I keep my domain registrar and just change DNS records?

Yes, that's exactly how it works. You don't need to transfer your domain registration. Just update the A record and TXT record at your current registrar to point to Lovable.