AI technology microchip background digital transformation concept. — Image by rawpixel.com on Freepik
New York: A bank uses biased artificial intelligence (AI) outputs in a mortgage lending decision. An insurance firm’s AI produces racially homogeneous advertising images. Users of an AI system complain about a bad experience.
These are just a few of the potential risks AI poses for financial institutions that want to embrace the emerging technology, according to a series of papers released last Thursday.
The papers, by FS-ISAC, a nonprofit that shares cyber intelligence among financial institutions around the world, highlights additional pitfalls as well, including deepfakes and “hallucinations,” when large language models provide incorrect information presented as facts.
Despite those risks, the Financial Services Information Sharing and Analysis Center (FS-ISAC) outlines many potential uses for AI for financial firms, such as improving cyber defenses. The group’s work outlines the risks, threats and opportunities that artificial intelligence offers banks, asset managers, insurance firms and others in the industry.
“It was taking our best practices, our experiences, our knowledge, and putting it all together, leveraging the insights from other papers as well,” said Mike Silverman, vice-president of strategy and innovation at FS-ISAC.
AI is being used for malicious purposes in the financial sector, though in a fairly limited way. For instance, FS-ISAC said hackers have crafted more effective phishing emails, often refined through large language models like ChatGPT, intended to fool employees into leaking sensitive data. In addition, deepfake audios have tricked customers into transferring funds, Silverman said.
FS-ISAC also warned of data poisoning, in which data fed into AI models is manipulated to produce incorrect or biased decisions, and the emergence of malicious large language models that can be used for criminal purposes.
Still, the technology can also be used to strengthen the cybersecurity of these firms, according to the reports.
Already, AI has shown to be effective in anomaly detection, or singling out suspicious, abnormal behavior in computer systems, Silverman said.
In addition, the technology can automate routine tasks such as log analysis, predict potential future attacks and analyze “unstructured data” from social media, news articles and other public sources to identify potential threats and vulnerabilities, according to the papers.
To safely implement AI, FS-ISAC recommends testing these systems rigorously, continually monitoring them, and having a recovery plan in the case of an incident. — Bloomberg
