Investment race
2023 witnessed a surge in financial backing for Gen AI, highlighting the growing recognition of AI's potential in the business world. Microsoft's multibillion investment in OpenAI underscored the strategic importance of AI in future tech developments. Accenture's $3 billion investment illustrated the escalating interest in integrating AI into business solutions. Nvidia's stock price leapt, driven by the demand for GPUs essential in AI model training and inference, further validated the sector's economic vitality. Salesforce's investment in HuggingFace, valuing it at over $4 billion, showcased the competitive landscape and the value seen in AI startups. These investments indicate not just confidence in AI's future but also a recognition of its critical role in shaping the next wave of technological advancement.
Competition heated up
This year was pivotal for AI innovation, with numerous high-impact product launches. Meta's launch of Llama 2 signaled increased competition by open-source AI. OpenAI continued to lead with GPT-4 and then GPT-4 Turbo, maintaining its gold standard status, while the introduction of ChatGPT Plus, Dall-E 3, and multimodal features reflected the diversification and expansion of AI applications. OpenAI's Custom GPTs and ChatGPT Enterprise, highlighted the race for more personalized and powerful AI tools. Elon Musk's entry into the AI chatbot arena with Grok as well as Google's launch of Gemini both posing a direct challenge to OpenAI's dominance, signifies the intensifying competition among tech giants. The advancements in AI image generation, as seen with Midjourney's updates and Baidu's entry, underscored the rapid evolution of AI capabilities. Copilot for Microsoft 365, Google Duet, Zoom AI, Salesforce Einstein, Meta AI, and Snapchat MyAI demonstrated AI's growing integration into everyday tools and services for work and pleasure. These launches collectively illustrate a robust and vibrant landscape of AI innovation, poised to redefine various facets of technology and business.
We changed the ways we acquire knowledge
The year 2023 witnessed a remarkable shift in the tech community's approach to information acquisition, marked by a significant decrease in Stack Overflow's traffic, largely due to the emergence and adoption of Gen AI coding tools. This change underscored a new trend in how technical information is sought and shared, indicating a move towards AI-driven solutions for coding queries and challenges. In parallel, this inclination towards AI solutions was mirrored in the broader search engine domain by Google and Bing new players such as Perplexity AI, and acquisitions e.g. Neeva by Snowflake.
Yet, last year marked not only a shift in how technical knowledge is acquired but also a significant transformation in the broader landscape of workplace productivity, thanks to the integration of Generative AI. Studies like the "Navigating the Jagged Technological Frontier" from Harvard Business School and Boston Consulting Group view study demonstrate the profound impact of AI tools like GPT-4 in enhancing task efficiency and quality in knowledge-intensive work. This study reveals a significant increase in productivity, with consultants using AI completing tasks 25.1% more quickly and improving the quality of results by over 40%. Additionally, research by the National Bureau of Economic Research illustrates how customer support agents employing AI tools saw a 14% increase in productivity, with the most significant benefits accruing to less experienced workers. Similarly, another study highlighted by the MIT Sloan School of Management found that the use of Generative AI could enhance the performance of highly skilled workers by up to 40%. These findings collectively paint a promising picture of AI's capacity to augment human effort across various skill levels and job functions, heralding a new era in knowledge work where AI's role is increasingly pivotal in driving efficiency and innovation.
Gen AI reliability and efficiency surged
In 2023, the reliability of Large Language Models (LLMs) witnessed remarkable advancements, as evidenced by reports from various leaderboards including The Big Benchmarks Collection by Hugging Face, Toloka's LLM Leaderboard, LLMPerf by Anyscale, and benchmarks such as MMLU and HellaSwag. These improvements were characterized by enhanced reasoning abilities and general knowledge, with models demonstrating increased proficiency in handling complex questions and commonsense reasoning. Technical progress was seen in tokenization techniques, attention mechanisms, positional encodings, and activation functions, leading to more efficient and accurate language processing. Benchmarks like HaluEval and Vectara's model specifically targeted hallucination detection, enhancing factual consistency in LLM outputs. Application-specific frameworks like SQLEval demonstrated significant progress in fields like SQL generation, where accuracy reached new heights through better context integration and testing architectures. Additionally, advancements in normalization techniques and distributed training methods contributed to faster convergence and stability in model training.
LLM dependency on static data that limits its ability to provide factual accuracy as well as domain-specific answers was resolved by the advancement in Retrieval-Augmented Generation (RAG) significantly advanced by LangChain, Llama Index and OpenAI. LangChain enhanced retrieval performance by employing distance-based vector database retrieval and offering over 60 vector store integrations for varied configuration options. Llama Index facilitated the construction and evaluation of RAG systems with tools for generating question-context pairs, crucial for assessing the retrieval accuracy and response quality. OpenAi introduced Assistants API which allowed to overcome file limitations and reduce token usage costs.
Advancements in reliability came hand in hand with the overall LLMsl efficiency increase through 2023 due to techniques e.g. post-training quantization. Key developments included the use of lower-bit quantization for enhanced computation, exploration of integer versus floating-point formats in advanced hardware, and studies on quantization effects on model performance. Techniques like LoRA-Fine-Tuning-Aware Quantization (LoftQ) achieved high compression rates while maintaining accuracy, aiming to make LLMs more deployable on devices with limited computational power. The advancements on the research side have been significantly bolstered by the contributions of companies such as CentML, Together AI, and Fireworks AI. CentML improved AI model deployment with cost and performance optimizations. Together AI enhanced model development with a scalable cloud platform and supported open-source AI initiatives. Fireworks AI focused on a fast LLM inference platform, optimizing code completion and making AI-powered coding more accessible.
These advancements collectively enable more nuanced and contextually relevant responses from LLMs, all while improving scalability.
LLMs became multi-modal
Multi modality enabled LLMs to understand and generate content across different formats, enhancing their functionality in areas like education, healthcare, and the creative arts. Multi-modal LLMs can generate narratives from images, translate spoken words to text, or create visual content from text descriptions. OpenAI enabled image and voice inputs to ChatGPT and generation of image outputs, while Google raised the bar with deeper image integration into Bard.
Looking at the evolution of Midjourney below, we might want to adjust the usual assumptions we hold for the pace of improvement in these models.
Navigating Ethics, Risks, and Policies
In 2023, the Generative AI (Gen AI) field experienced notable ethical advancements amid growing concerns over its rapid development. Elon Musk, Steve Wozniak and others, initiated a petition to halt the creation of AI models surpassing the capabilities of GPT-4, highlighting the apprehension surrounding the potential risks of such advanced technologies.
Major tech companies were the first to actively address ethical issues. Meta required advertisers to disclose AI involvement in digitally altered ads. Initiatives like OpenAI's Superalignment focused on controlling smarter AI systems and Anthropic published a Responsible Scaling Policy to require increasing safety and security as model capabilities improve. Samsung responded by restricting the use of Gen AI tools among employees, citing security concerns and potential misuse. Alongside the above, the Frontier Model Forum, including Google, Microsoft, OpenAI, and Anthropic, launched to promote safe AI development.
In response to rapid advancements in Gen AI, 2023 saw significant legal and state policy developments, with the European Union finalizing the Artificial Intelligence Act that established comprehensive regulations for AI technologies. The White House issued an executive order on AI that set safety measure and security standards for developing advanced AI systems. The UK hosted the first Global Summit on AI Safety, aimed to foster an inclusive debate and involved various stakeholders, including governments, academics, civil society groups, and leading AI companies.
