Large Action Models (LAMs): A New Step in AI for Understanding and Doing Human Tasks : Emily Rosemary Collins
by: Emily Rosemary Collins
blow post content copied from Be on the Right Side of Change
click here to view original post
Definition: A Large Action Model (LAM) is an advanced artificial intelligence system designed to perform human-like tasks within digital environments. Utilizing neural networks and symbolic reasoning, it interprets and executes complex commands in applications, like web navigation or online transactions, by directly modeling and understanding the logic and structure of these interfaces.
Artificial Intelligence (AI) has a new development called Large Action Models, or LAMs. These models are an advanced form of Large Language Models (LLMs), which many of us know about. LLMs create text by guessing the next word based on input.
LAMs go further. They turn LLMs into ‘agents,’ software that can do tasks alone. This means they can do more than just answer questions; they can work towards a goal. This is a big change because it mixes LLM’s language skills with the ability to do tasks and make decisions on their own.
LAMs are built to mimic the way applications and human actions work together. They use neuro-symbolic programming to do this without needing text. This is a complex area, and we don’t have complete access to these models yet.
LLMs and LAMs are both AI models, but they’re different. LAMs can connect to real-world systems, like IoT devices. This lets them do physical tasks, control devices, gather data, and handle information. They can automate whole processes, talk to people, adapt to changing situations, and work with other LAMs.
LAMs are powerful for several reasons.
- First, they understand complex human goals and turn them into actions.
- Second, they can smartly interact with the world, including people and changing situations.
- Third, they connect with real-world systems.
- Lastly, they turn generative AI from a simple tool into a helpful partner in real-time tasks.
LAMs have many potential uses. In healthcare, they could change patient care with better diagnostics and treatments. In finance, they could help with risk analysis, finding fraud, and making algorithm-based transactions. In the automotive industry, they could improve self-driving cars and safety systems.
Rabbit R1 Device – What’s the Hype?
Application: A practical example of LAMs is the Rabbit r1 device, selling for $199. It’s a small device, about half the size of an iPhone, with a touchscreen and a rotating camera. It has a scroll wheel for easy navigation. The Rabbit r1 uses Rabbit’s AI operating system (OS) and LAM. This lets it understand and mimic human actions on technology interfaces. Rabbit’s advancement shows how online interactions can become easier without needing apps.
LAMs are set to greatly impact AI’s future. They turn language models into ‘agents’ that can do tasks, making AI a real-time action partner. Products like Rabbit are already using LAMs, opening up many new possibilities. This marks a major shift in AI development.
For example, the idea of LLM-based operating systems has been proposed recently by Karpathy, Tesla’s former head of AI:
This is just the beginning. The idea of natively integrating LLMs and LAMs into devices is coming!
Stay updated on this exciting area of AI technology by joining our AI newsletter and downloading our Python cheat sheets:
Technical Introduction to Large Action Models (LAMs)
LAMs embody a transformative approach to human-computer interaction, shifting the paradigm from traditional graphical user interfaces to more intuitive, action-oriented models.
At the core of LAMs is the objective to mimic and execute human actions within computer applications, thereby creating a more natural and efficient user experience.
Unlike their predecessors, Large Language Models (LLMs), which mainly focus on understanding and generating text, LAMs are designed to understand and replicate complex user tasks, ranging from web navigation to application-specific operations. This capability is achieved through a novel neuro-symbolic model that blends the structured nature of symbolic algorithms with the adaptive learning prowess of neural networks.
The need for LAMs arises from the inherent limitations of conventional AI models in comprehending and interacting with application interfaces. Standard language models, for instance, struggle to fit the representation of web applications within their contextual understanding due to the verbose and noisy nature of these applications. LAMs address this challenge by learning directly from user interactions, bypassing the need for rigid Application Programming Interfaces (APIs) and focusing on action-oriented learning.
A crucial aspect of LAMs is their ability to learn actions through demonstration, allowing them to adapt to variations in interfaces and execute tasks reliably. This approach not only enhances the model’s efficiency and accuracy but also ensures explainability and simplicity in task execution. By understanding the structural and logical composition of applications, LAMs can perform actions that align closely with human intentions, thus bridging the gap between users and digital services.
The integration of LAMs extends to various practical domains, including web navigation and automated processes in mobile and desktop environments. The model’s competitive edge is evident in tasks like web navigation, where it demonstrates superior accuracy and latency compared to existing methods. This proficiency is underpinned by a technical stack that combines transformer-style attention, graph-based message passing, and program synthesizers guided by demonstrations and examples.
“We believe that in the long run, LAM exhibits its own version of “scaling laws,” [3] where the actions it learns can generalize to applications of all kinds, even generative ones. Over time, LAM could become increasingly helpful in solving complex problems spanning multiple apps that require professional skills to operate.” — Rabbit Research
If you want to learn more about scaling laws, check out our blog article:
AI Scaling Laws – A Short Primer
Frequently Asked Questions
What exactly is a Large Action Model (LAM)?
A Large Action Model (LAM) is an advanced AI system that can carry out human tasks on computer applications. Unlike just providing information or instructions, LAMs can actively perform tasks like navigating websites, filling out forms, or online shopping. They combine neural networks and symbolic reasoning to model and execute various application tasks.
Who created the first LAM?
The first LAM was developed by Rabbit Research Team, a pioneering AI company. They introduced Rabbit R1, a device that uses LAM technology to execute complex tasks on applications using natural language commands.
How is LAM better than other AI models?
LAMs have several key advantages:
- Accuracy: They perform tasks with high precision.
- Interpretability: LAMs can explain their actions and logic in a clear manner.
- Speed: They complete tasks quickly by avoiding extensive data processing or multiple inference steps.
- Simplicity: LAMs can handle complex tasks with simple commands and require minimal training.
Can you give some examples of what tasks a LAM can perform?
Sure! LAMs can perform a range of tasks across different applications:
- Flight Booking: For instance, booking a flight on Kayak by specifying details like destination, date, and budget.
- Form Filling: Filling out forms on Google Docs with required info and formatting.
- Grocery Shopping: Shopping on Instacart by adding items to the cart and checking out.
- Playlist Creation: Making a playlist on Spotify based on genre, mood, and artists.
- Content Summarization: Generating summaries of Wikipedia articles by identifying key points and keywords.
January 25, 2024 at 02:37AM
Click here for more details...
=============================
The original post is available in Be on the Right Side of Change by Emily Rosemary Collins
this post has been published as it is through automation. Automation script brings all the top bloggers post under a single umbrella.
The purpose of this blog, Follow the top Salesforce bloggers and collect all blogs in a single place through automation.
============================
Post a Comment