How will AI change our jobs?

Brian

Jan 30, 2024 — 15 min read

I think it's safe to say that the biggest buzzword of 2023 was generative AI. ChatGPT has more than 180 million subscribers and nearly 100 million weekly active users. Everyone, from the general public to the media, is talking about generative AI. This makes me consider trying it myself, but I'm at a loss for what questions to ask or how this technology could be applied to my job or company. After a brief test, the results don't seem as impressive as I thought, and it seems like most people have concluded that AI still has a long way to go and have simply forgotten about it.

In this article, I'm going to talk about how AI agents are changing the business world and how they will affect our personal work and future, focusing on large language models (LLMs) like OpenAI's GPT-4.

(Throughout the article, I use the terms AI, AI agent, generative AI, and LLM interchangeably for ease of understanding. Technically, they all have slightly different meanings, but I'm just using them in a way that's easier to write and understand.)

How does generative AI impact knowledge workers?

The impact of AI on the knowledge worker job market is more significant and widespread than one might think. A recent study found that after the launch of ChatGPT and Midjourney, the number of jobs and earnings of copywriters and graphic designers on major online freelance platforms dropped significantly. This suggests that the products of generative AI are being subjected to direct competition with human-created output, directly impacting their jobs.

OpenAI estimates that new advances in AI pose the greatest risk to the highest-paid jobs, with high-wage workers being about three times more at risk than low-wage workers, according to its analysis. This indicates that AI could impact not only simple and repetitive jobs, but also jobs that require specialized knowledge.

The emergence of ChatGPT has significantly triggered discussions about AI strategy in many corporate boards, with AI adoption rates increasing even more dramatically. As generative AI solutions prove their value and become embedded in a wider range of business processes, their application is likely to expand further.

We need to recognize that AI is transforming jobs, not just replacing them. By automating repetitive tasks, AI enables employees to spend more time on strategic and creative work, which, in turn, improves productivity and efficiency, fueling business growth and creating new jobs. AI technologies are already fundamentally changing the business landscape, reshaping workforce structures and jobs across industries.

Does generative AI really help productivity, and how?

Github Copilot, a leading AI solution, helps software developers speed up the development process and reduce the likelihood of errors. According to recently published data, developers who used Copilot experienced more than a 70% improvement in code clarity and writing speed. AI assistance has been shown to be especially useful for novice developers, helping them learn and improve their coding skills faster. This shows the potential of AI to not only improve productivity and quality, but also serve as a learning tool.

A study of users of Microsoft's AI solution, Copilot, found that: 1) users experienced an average of 27% speedup in tasks such as conducting meetings, writing emails, and creating documents. 2) In an evaluation by an independent panel of experts, emails written with Copilot were rated 18% better for clarity and 19% better for brevity. 3) AI-enabled users were also 27% faster at gathering and organizing information from multiple sources.

Harvard Business School recently studied the impact of GPT-4 on productivity and quality of work among employees at The Boston Consulting Group (BCG). The study found that BCG consultants assigned to use GPT-4 while working on a series of consulting tasks were more productive than their peers who did not use the tool. Specifically, the AI-assisted consultants performed their tasks 25% faster and handled 12% more workload, and their quality of work was rated 40% higher than their non-AI peers.

One of the interesting findings of the study was that users in the bottom 50% of the evaluation saw a 43% improvement in performance by using AI tools, while the top users saw a 17% improvement. This means that all employees, regardless of skill level, benefited from utilizing AI, but especially those who were previously rated low. This suggests that AI can close the gap between people's ability to perform tasks and improve the overall quality of work for everyone in the organization.

The study also describes two groups of people who differed in the way they collaborated with AI. The first group, dubbed "cyborgs," worked closely with the AI, constantly generating, reviewing, and refining answers, while the second group, dubbed "centaurs," split the roles, focusing on their own areas of expertise and giving the AI subordinate, supportive tasks.

Finally, the study found that the role and effectiveness of AI did not appear to be the same in all areas of work. In particular, for some tasks that require a deeper understanding of the problem or more nuanced judgment in solving it, AI did not deliver the improvements noted above. This shows that while AI has the potential to improve productivity and quality in many areas, it cannot fully replace the deep cognitive abilities and expertise of humans.

So why aren't we there yet?

Numerous studies and articles are already talking about a world transformed by AI. So why am I not taking advantage of AI? Why does no one around me seem to be using AI? What's the problem?

As I talk to different people about using AI in their work, I realize that there is a huge gap in people's understanding and ability to use AI. People have varying perceptions of the role, capabilities, and limitations of generative AI, as well as diverse approaches to its utilization and task delegation. This leads to a huge gap in the ability to utilize AI. If this trend continues, the disparity in output between individuals will only widen.

Many people still view LLMs like GPT as simple search engines or knowledge databases. As a result, it's easy to underestimate the power of LLMs when they don't have information about the latest events or can't provide accurate answers about specific expertise.

In fact, we should understand LLM models like GPT as reasoning engines. Reasoning is the mental task of analyzing observed information or collected data to discover regularities or patterns, make predictions about future events, or draw conclusions for deeper understanding. This process is essential for making connections between information, coming up with solutions to complex problems, and generating new ideas. In a knowledge-based society, these reasoning skills are key to the daily work of a wide range of professionals, including scientists, engineers, doctors, lawyers, business analysts, and others. Studies have shown that in many fields, GPT-4 reasoning and decision-making skills are already higher than those of the average person. LLMs can play an important role in creativity, innovation, analysis, persuasion, and more.

Large Language Models (LLMs) alone are not enough.

Why does the LLM always seem to give a dumb (insufficient) answer whenever I use it? This is because LLMs lack both short-term and long-term memory and do not have appropriate reference materials or guidelines. LLMs have a limited context window, allowing them to focus only on brief information and moments. If you try to process too much information at once, you will not be able to process it properly and will lose your overall understanding. Without proper guidance, LLMs might start talking haphazardly, fabricate nonexistent facts, forget important details, or remember things incorrectly.

However, what makes LLMs potentially powerful is their ability to utilize a variety of tools and knowledge bases based on their reasoning capabilities. This allows LLMs to make better, more informed decisions and take action using a variety of tools. LLM performance improves dramatically. Of course, choosing the right data sources and the right tools are essential to making LLMs work for you. LLM is not just a search engine, it's an inference engine.

How can we make AI do its job better?

The instructions we give to an AI are known as prompts. However, depending on which prompt you give it, the results you get can be completely different.

Let's switch gears for a moment. When you first started your job as an entry-level employee or intern, you probably encountered many various mentors. Think back to when you first received your work assignment and started working. A good mentor will define the purpose of the task well, understand your abilities and experience, advise you on how to execute it, and let you know in advance where you might make mistakes. They also clearly communicate their expectations for specific deliverables. In this case, it's much easier for a talented subordinate to understand the purpose of the work and do their best to deliver the required results. On the contrary, a bad mentor provides unclear definitions of tasks, lacks understanding of the other person, and asks for results in an abstract manner. Naturally, it's hard to get a good result with such a work order. (The bad mentor might nitpick why the results are only this much...) In fact, collaboration with AI is very similar to this. Obviously, the good mentor here can write much better prompts and get better results. Of course, it would be ideal if you had a subordinate or an AI that can understand gibberish as if it were clear instructions, but that still seems difficult."

1) People who know the role of AI as an inference engine and understand its capabilities and limitations can design more sophisticated prompts. 2) People who know what they need to input and what they want to get out of it will be able to write better prompts. In this sense, people who are more knowledgeable and experienced in their domain are more likely to write better prompts. 3) People who have experience working with diverse teams are generally more likely to use AI and chatbot platforms effectively. They are better at guiding AI to more accurate and useful responses. 4) The ability to structure your work also has an important impact on prompt design. People with a structured mindset are able to clearly set goals, design a course of action, and effectively communicate the information they need to the AI. When solving complex problems, this allows you to break down each part of the problem and craft prompts accordingly to get the most out of the AI's capabilities.

Users can understand the AI's responses in full context and, if necessary, ask for more information or follow up with more specific questions. This approach (similar to the cyborg type mentioned above) is crucial to getting better results from interactions with AI. You can play to the AI's strengths in analyzing the data it provides, strategizing, and making decisions. This allows you to make the most of AI's responses and work more efficiently and effectively. In fact, this ability is critical not only for AI-enabled work, but also for collaboration between human coworkers.

Data Safety Issues

ChatGPT offers good value for money. A free version is available, and with the Pro version, although there are some time limitations, you can continue to use GPT-4. Directly connecting to the API for the same amount of tasks would incur much higher costs. Why is ChatGPT cheaper?To borrow a famous saying from the industry, "If you're not paying for the product, you are the product." This means that if you're not paying for it, your data is the product.

LLM companies explicitly state that their web-based products, such as ChatGPT and BARD, collect user conversation data and use it to train their models. These platforms rely on user-supplied conversation data to address data shortages and to collect a variety of use cases to improve their models. If you enter sensitive information into ChatGPT, it can be stored for as long as they want and used in the training process in any way they want. This means that there is a possibility of sensitive information being leaked. In fact, a recent case was reported where ChatGPT exposed email addresses used in the training process by including them in the output.

The above is probably a case where someone's email was used for training and the results happened to look exactly the same. Of course, this also means that your work input into ChatGPT could be used as training data and appear in someone else's output as shown above.

Alternatively, you can install the open-source LLM in an on-premise environment. This way, you can manage all of your data on your own servers. However, installing and securely maintaining such a system can be more expensive than you might think. Open source LLMs still have a significant performance gap over commercial models. While many benchmarks point to recent improvements in the performance of open source-based LLMs, it is very likely that the actual performance in real-world usage may be lower than what benchmark scores reflect. This is especially true in areas such as access to external knowledge and the use of tools. The gap is even more pronounced when prompting and receiving results in languages other than English. This gap is unlikely to close in the near term.

The next option is to use GPT as an API. OpenAI, Microsoft Azure, Google, and Anthropic all offer LLMs as APIs, and they all specify that the data is not used for training purposes. By sending only the data you need through the API, you don't have to share the full context. This cannot be said to be completely safe. However, it's worth remembering that many organizations had similar concerns when AWS, GCP, and other cloud services were first introduced. Eventually, the performance, cost, and overall utility outweighed the initial concerns. If you're already using AWS, GCP, Slack, Notion, Google Docs, etc. for your business, the additional concerns about using LLM APIs may be unnecessary. (There's no significant difference)

In conclusion, the most effective and accessible way to use LLM at this point is to develop your own solution utilizing the APIs in GPT, Claude, and Gemini, or use a service that indirectly utilizes LLM APIs, such as Kompas AI. If you choose to develop your own, be prepared to deal with a variety of technical challenges, such as using Retrieval-Augmented Generation (RAG), system prompts, memory, and tools. To get the most out of an LLM, you'll need to do a lot of fine-tuning (often very labor-intensive), which is not well understood across many disciplines and requires a lot of expertise (experience through lots of trial and error).

So how does this apply to our work?

Several studies have shown that LLMs excel in the following areas: 1) creativity and innovation, 2) analyzing, persuading, and managing, 3) providing initial feedback on work, and 4) helping with decision-making and providing a second opinion. For example, a writer or designer can use an LLM to generate multiple versions of a text or design concept and then select or refine the best one. Business analysts and data scientists can also use LLM to extract insights from data and analyze them further to make better, more informed decisions faster.

As mentioned above, LLM is a reasoning engine, not a search engine. By ensuring that your prompts reflect your clear goals, how you want to proceed, and the expected outputs, and by connecting the right knowledge and tools, you can perform complex and varied tasks and get high-quality results that you can use immediately in your work. Here are a few use cases

Here are a few examples of use cases (a bit long)

1) Swiftly creates straightforward market competition reports.

2) Conducts strategic analysis with various strategic frameworks.

3) Analyzes the potential market impact based on hypothetical scenarios.

4) Processes and visualizes data, albeit with some hiccups, but the results turned out well

In addition to the aforementioned tasks, an AI agent:

Offers valuable insights for business decision-making processes.
Assists with complex research.
Processes and analyzes complicated data.
Proposes creative plans and strategies.
Supports both coding and debugging.
Enhances user experiences with UX writing assistance.
Screens a multitude of resumes automatically.
Writes and distributes OKRs and KPIs.
Analyzes data from diverse documents to compile reports.
Organizes complex thoughts and aids in setting priorities.

These examples highlight the broad range of tasks an AI agent can handle, among many other possibilities.

By the way, most of these AI agent examples here cannot be properly executed in ChatGPT. First of all, you need to be able to write the various prompts yourself, and if you try to run more complex tasks, ChatGPT will often reject them (due to resource constraints), and even if it does, it will most likely time out at some point and crash.

Running a sophisticated AI agent requires unrestricted operational resources and an infrastructure that can handle multiple iterations. One must also overcome the 60-second execution constraint of traditional code interpreters. As for search engines, Google's results are significantly better than Bing's. (In non-English-speaking regions, Bing's search results are almost disastrous.) To implement these features, you need to develop your own infrastructure for LLM operations, set up your own AI agents, or use a service like Kompas AI. The aforementioned outputs were all generated using AI agents based on GPT and Claude through Kompas AI.

I keep mentioning Kompas AI because I am developing it. To give a brief introduction to the product:

Kompas AI offers well-designed AI agents for various purposes, allowing users to easily and quickly derive excellent results.
Kompas AI provides AI agents that solve much more complex and difficult problems based on Google integration, very high iteration counts, support for various LLM models, and a powerful Advanced Code Interpreter.
There are also features for team-based users, so you can create customized AI agents and share them among team members.
Best of all, Kompas AI has strict policies on data governance and privacy, ensuring that no data is used for training. Users have full control over their data.

If you're interested, sign up for a free trial at https://kompas.ai.

What if you're not using AI yet?

According to a recent report from Salesforce, 28% of respondents stated they are already using AI in their work. More than half of them, 51%, expect AI skills to increase their job satisfaction, and 44% expect to earn a higher salary than their colleagues without AI skills.

At the same time, however, many companies have not yet provided clear guidelines on the potential and ways to apply AI in work. Among those who use ChatGPT for work, 55% report using AI services without formal permission from their company. It seems that business leaders still see AI as a distant and incomplete future technology. AI has shown significant advancements throughout 2023, and more changes are expected in 2024. Even now, we need to understand the capabilities and limitations of AI tools and have appropriate usage policies in place. We need to ensure that our members are able to fully utilize new technologies to increase productivity while keeping their data secure.

Finally, to quote World Economic Forum economist Richard Baldwin, "AI won’t take your job, but someone who uses AI will" I would add to this, "AI won't ruin your company, but a company that effectively uses AI will".

It's never too late to start, even when you think it is. Compared to the overall changes we are going to experience, we are still at the very beginning of change.

There may be inaccuracies or omissions in this content due to my limited experience and knowledge. Please point it out so I can improve.

Reference:

Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality

The public release of Large Language Models (LLMs) has sparked tremendous interest in how humans will use Artificial Intelligence (AI) to accomplish a variety o