Generative AI and LLM's - Excitement and Panic
What do you feel
Disclaimer: The following articles are my own comments and based on my own research (links below) and have no bearing on my employer. Any reproduction of this article needs explicit permission from the author. Other than the history of evolution, rest of the contents are strictly my opinion based on research.
Today everyone, even the ones who are not on LinkedIn, are talking about ChatGPT. The world is changing around us and it’s strange to see technology evolving at such a fast pace. I was in my high school when the Internet revolution began and I was in the spectrum of time with much excitement within my fingertips. Ability to connect with strangers through AOL, ability to email with long distance friends and get an instant response, ability to have a phone call over the internet, ability to access information as quickly as I can. I felt the enthusiasm and energy and the decades which followed through proved how valuable this was. The landscape changed with the Internet and we are so grateful for all that it has offered and the lives it has changed.
In this essay, I would like to give a primer of what Generative AI and LLM models are. I am no means an expert, like the rest of the world I am watching this unfold and this content is based on what we know as of today’s date (Feb 17, 2023).
I will cover the below:
-> What is LLM?
-> What is Generative AI?
-> History of evolution and the Hype
-> Who are the key players in the market?
-> How would enterprise behavior change?
-> Controversies surrounding this area
-> Predictions for 2023
Generative AI:
Artificial Intelligence is a field of study where the machine understands and reacts by mimicking human behavior based on the data the model has been trained on. Generative AI is algorithms which have the ability to create content based on the data the model is trained on, and also the ability to generate new and unexpected content. The content can be speech, text, code, image, video, 3D objects, and decisions for games.
LLM:
LLM (Large Language Models) is a type of model in Generative AI which has been fed large amounts of text data from across the internet wikipedia, scientific articles, books, research papers, blogs, forums , websites etc., to train so it can generate new content similar to the one it has been trained on. The larger the model, the more performant the model would be. These models can solve Natural Language use cases such as question / answering, summarization, writing new content, generating code, and performing sentiment analysis.
Some examples of these models include GPT-3, BERT, T5, XLNet.
Now, let’s uncover a bit of the history of the models to understand how this has evolved.
Natural Language Processing (NLP) is a field of interest which has always gained the interest of researchers as we humans tend to use language to communicate. NLP has two main areas of interest: Natural Language Understanding (NLU) and Natural Language Generation (NLG).
The evolution of NLP dates back to the 1950s with a heuristic approach. Then, we evolved to a more machine learning and a deep learning approach.
Credit goes to Google, the most of the focus to Natural Language has come from them given the need of sprawling across the Internet to provide a better search experience.
The following image summarizes the evolution of these models based on the below blog references.
https://huggingface.co/blog/large-language-models
https://code.google.com/archive/p/word2vec/
Google DeepMinds Chinchilla is the largest model with 1 Trillion parameters used to train the model as of today.
As of now, we have only discussed language models. There are similar models based on images (DALL-E) and other content types which exist today, but this can be discussed later.
Who are the key players in the market?
As we saw above, companies such as Google, OpenAI, Microsoft, Amazon, NVIDIA, and IBM have all produced large models for usage. The large players in the market can only afford the resources needed for such large models today. We have seen companies investing billions of dollars in this. Microsoft just announced $10B OpenAI investment. This is besides the $1B investment in 2019. Google reportedly has invested $120B since 2016 in this space and has also announced a recent $300M investment on Anthropic (founders from Open AI).
However, small niche startups are using these models to build new products for mass adoption.
How would enterprise behavior change?
For Enterprises, this is an exciting time. No matter any industry we are from the world, we know it will definitely change. But the level of change is something we are going to be all watching as it unfolds. Based on Gartner Hype Cycle, we are on the rise or at the peak of the hype.
Every organization today irrespective of Industry is most likely looking cautiously on how they could improve productivity, collaboration and efficiency in the market.
Imagine a world where
… an employee working on a piece of code, able to generate the code with Generative AI and test with another set of data generated. This might not be perfect the first time around but you could continue to ask the Generative AI to create a more fine tuned one thus resolving a humongous amount of time.
… a Customer Support Representative having the ability to get an email drafted with very less amount of time involved?
… your teams have access to all the information in silos across your platforms, the ability of impact it would create
… autocreates content for learning for your learning platforms
… and many many more, we are just starting!
In my opinion, even though this seems very far-fetched particularly when some organizations are yet to evolve from green screens. I would think many companies with giant Enterprise market share such as Salesforce, SAP, Adobe and others would start integrating to their platform pretty quickly. In fact, we saw this pretty quickly from the ChatGPT integration to MSFT products and the continued integrations we see in Google Workspace.
**Controversies **
Would it replace Google Search ?
We need to remember the discussion we have had till now focuses on generating content on what the AI thinks is appropriate answer based on the data it is trained vs Google Search serves the absolute information with the link. It might use AI to do search ranking / scoring for the top content but it does not use AI to generate the data. There lies the difference between a bot and a tool. We might see more conversational aspects of search both in Google Search and Bing but it is very much unlikely to replace the absolute information with generated content particularly when training these models is a costly effort.
Other mishaps
Call for Testing transparency
These tests and problems are possible in a world when the technology is not mass adoption ready but has the hype that couldn’t meet the standard. Google in adherence to its Responsible AI principles have been cautious when they are releasing Bard (competitor to ChatGPT) to a trusted tester audience. Every organization is forced to come up with their own set of standards and practices and most often dollar signs come in the middle of them. Microsoft making this choice was unfortunate but I am glad they are taking steps to revert back some of the steps
Call for Training Transparency
Google again set the standard by releasing their Representational Bias Analysis and the artifacts such as model cards and artifacts paves the way for other companies to follow.
- Laws should be enforced to encompass fairness, privacy and interpretability of AI applications
- Organizations such as WHO has enforced Health AI ethics
- UK Government has enforced Financial AI Ethics
- UK Government has published their Data and AI Ethics Framework
- NIST (National Institute of Standards and Technology) released World Wide Web and Information Security guidelines in 1998 which most organizations adopted as part of their Cyber security strategy. However, currently there are no set guidelines available for Ethical AI in NIST other than the page here.
With all the advancement we have, these controversies are concerning and need to be handled more with a holistic approach than letting every organization decide the standard for themselves.
Predictions for 2023
- Every organizations sales conference keynote will have Generative AI
- Small niche AI product based apps will start to pop. Example : Lensa AI, jasper.ai, databloom.ai
- Large AI players will continue to compete in this space by integrating to the platforms
- Many LinkedIn profiles will be updated with Generative AI, LLM architects and industry experts
- Job descriptions starting to ask expertise for 10 years experience
In conclusion, my teenage daughter said this “The ones who are worried about ‘what this is bringing’ are the ones who were born before mobile phones existed”. She could be true.
At this time, I am overly excited for the Enterprise AI innovations waiting to happen and looking forward to being on the envisioning side. I see a world which does not exist today with lots of opportunities in every Industry and every role we see in the Enterprise. Customer service , Sales and Marketing will be pioneers for most of these advancements.
However, my societal side is extremely in a panic with the rapid AI involvement and no guard riles around.
Please let me know in the comments if you have another topic in this area you would like to cover and what your feelings are.
Other references used
https://twosigmaventures.com/blog/article/the-promise-and-perils-of-large-language-models/
https://ai.googleblog.com/2017/08/transformer-novel-neural-network.html
https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html
https://www.sequoiacap.com/article/generative-ai-a-creative-new-world/
Note: ChatGPT was not used to write any of the content above.
If you have questions/comments/suggestions, please reach out to me kanch@cloudrace.info
