KiwiSaver remains outside the box for ChatGPT

ChatGPT has taken 2023 by storm. The open language model “chatbot” developed and launched by OpenAI on November 2022 has certainly generated mass interest. It is almost surprising how quickly it has been adopted by individuals and companies alike. Editor’s Note: Chat Generative Pre-trained Transformer (ChatGPT) is an artificial intelligence chatbot developed by OpenAl and launched on November 30, 2022. It is built on OpenAl’s GPT-3 family of large language models and has been fine-tuned (an approach to Transfer Training) using supervised and reinforcement learning techniques. ChatGPT has been trained to interact conversationally to deliver answers within seconds, from simple to complex ones. The chatbot could eventually challenge the dominant Google Engine in the search technology space. Many have been praising the innovative technology set to revolutionise the way in which we generate and source information. There are plenty of voices criticising the tech and making a case for a harmful impact on society, from educators unable to differentiate a student essay from ChatGPT-generated content to others citing hacking concerns. Evaluating financial information We decided to evaluate the use of ChatGPT to generate financial information. In many other topics, the chatbot’s weaknesses have been exposed through incorrect information being generated. KiwiSaver advice is our speciality and hence we wanted to see how the tech stacks up against registered financial advisors. We took a turn asking ChatGPT a few basic investment questions and then upped the ante with more complex ones. Thereafter, we asked for custom advice based on a unique set of financial circumstances. Up next, similar questions were asked with KiwiSaver investment as the overarching topic. Lastly, we evaluate just how much should you rely on ChatGPT for investment advice. Investment basics Oftentimes, when you search on Google for basic information, you can get swamped with too much surrounding and irrelevant content. ChatGPT will generate a single answer as concisely or as long as you want. You can even prompt how many words or bullet points within which you want answers. I started with a basic investment question: What are the best investment avenues available to everyday people? The answer was good as a starting point. It began by stating that investing will depend on an individual’s goals, risk tolerance, and length of time. Then, it presented six bullet points of investment options such as stocks, bonds, real estate, and high-yield savings accounts. It provided a brief description of each option including risk assessment as a factor. So far, so good. We asked about the tax burden while selling an investment property in New Zealand. The answer did not include the Bright-line Property rule, where you could pay no tax after owning a property beyond ten years. While the answer about the tax payable was correct in certain circumstances, no tax is to be paid in others. Here is an example that ChatGPT being a useful tool for surface-level basic information. However, you should not solely rely on it when investing in property, setting up KiwiSaver, or any other investment avenue. Interestingly, if I had not known about the Bright-line rule in advance, the confident and succinct answer could have fooled me. Custom Investment Advice Learning general information about finance and investment is one thing. An advisor is to whom you go to get tailored recommendations based on your specific circumstances and goals. Is ChatGPT able to keep up, match, or even replace your financial advisor? Here is what we found after asking for the top 5 best-performing stocks on NZX 50 in 2022. The chatbot has been configured with data up until 2021 and thus lacks knowledge beyond that period. As anything to do with investing changes in real-time, from regulations to valuations, this is a clear roadblock. While that may change in the future, if you need the very latest data, ChatGPT was not helpful. Testing the AI model We tried to trick the AI language model by creating a scenario where its name is Bill and he is the top financial advisor in New Zealand. I asked ‘Bill’ about the stocks in which I should invest $10,000. Judging from what we have learnt about the chatbot so far, it was unsurprisingly a generic and vague attempt. It is not programmed to give concrete financial advice. It does a reasonable job of describing complex financial terms in plain English, but nothing more. The flip side is that it answers in such an authoritative way that people can take incorrect information from it. Whether you ask ChatGPT for advice for real estate, stocks, KiwiSaver funds, or any other investment, you will not get far. KiwiSaver basics KiwiSaver is one of the most widely adopted investment channels in New Zealand. However, many Kiwis struggle to understand the scheme and hence can miss some benefits. We asked ChatGPT a couple of questions about KiwiSaver investment options. It generated six points. One of them was factually incorrect. Can you guess which one? It is in fact, Number Three. KiwiSaver investment returns are not tax-free. The profits generated from the returns are taxed based on the Prescribed Investor Rate (PIR). At National Capital, we have written a comprehensive article on how you can change KiwiSaver providers and why people do. KiwiSaver recommendations I wanted to know what ChatGPT would say about  KiwiSaver based on a set list of goals and scenarios and test its ability to generate answers comparable to professional KiwiSaver advice. Which is the best KiwiSaver fund that will help me in buying my first home? Unfortunately, I could not the language model AI to generate anything of substance. The answer was more of a disclaimer to seek financial advice rather than a concrete option. Remember Bill, New Zealand’s top financial advisor from the custom investment section? He is now New Zealand’s top KiwiSaver advisor faced with the same question. While Bill was not able to name the best KiwiSaver scheme, the recommendation was to seek stability through a balanced or conservative fund. It is worth pointing out that when asked directly, ChatGPT will not give KiwiSaver advice upfront. However, we can see the loophole to unlock more of its capabilities by the way you structure a question. The weaknesses It should come as no surprise that found some incorrect answers. To be fair, every financial statement made by the chatbot was followed by a reminder to seek professional advice. Despite its dangerously authoritative wording, we can see the added risk of inaccurate information. No one should rely on the information generated through ChatGPT as financial or KiwiSaver advice. Clive Fernandes is an Authorised Financial Adviser and Director of National Capital, a financial advisory firm that provides personalised investment advice, with a primary focus on KiwiSaver. Disclaimer: The above article is not intended to be personalised advice. It is general and may not be relevant to an individual’s circumstances. Before making any investment, insurance or other financial decisions, you should consult a professional financial adviser. A copy of Clive Fernandes’ disclosure statement is available on request and free of charge.