This is exactly the kind of use of artificial intelligence (AI) that the EU's proposed AI Regulations is intending to heavily regulate.
In what is expected to be the largest regulatory shake-up since data protection, the proposed AI Regulations will place (GDPR-style) compliance obligations on creators and users of AI on a sliding scale of strength depending on the risk posed by the AI.
Certainly any AI used as a key part of any employment decision-making process is likely to be considered towards the highest risk use cases of AI.
Two unions are taking legal action against Uber, alleging that software used to verify drivers' identity is racially biased, and the firm has unfairly dismissed drivers. The unions claim the Microsoft facial verification system used is worse at identifying people with darker skin. Uber says it protects passengers by ensuring the right person is driving, and there is "robust human review". Microsoft declined to comment on an ongoing legal case.