Can I remove a design system from a project?

Yes, in Project Settings → Connected Design Systems, click the system you want to remove and select 'Disconnect.' The project loses access to that system's components and styles immediately. Components already using the system won't break, but you won't be able to use new components from that system.