Head over to our on-demand library to view sessions from VB Transform 2023. Register Here
True to the rumors and advance reports, Meta Platforms, the company formerly known as Facebook, today unveiled Code Llama, its new generative AI large language model (LLM) designed specifically for programming — and like the more general-purpose LLaMA 2, it’s open source and licensed for commercial use.
Code Llama is “designed to support software engineers in all sectors — including research, industry, open source projects, NGOs, and businesses,” Meta says in its blog post announcing the models.
The tool immediately becomes a major rival to OpenAI’s Codex (powered by a modified GPT-3), the Codex-powered Github Copilot from Microsoft, and other coding-specific LLM assistants such as Stack Overflow’s OverflowAI.
VB Transform 2023 On-Demand
Did you miss a session from VB Transform 2023? Register to access the on-demand library for all of our featured sessions.
Building on that analogy, the family includes three main members: a 7-billion, a 13-billion and a 34-billion parameter model, each trained on 500 billion tokens. The smaller models are designed to run on fewer GPUs (the 7-billion model can run on a single one), a beneficial attribute given the rumored scarcity in this critical piece of hardware at the moment, and Meta says both are faster than its 34-billion big model.
All models support up to 100,000 tokens for their prompts. This means “users can provide the model with more context from their codebase to make the generations more relevant,” according to Meta.
The LLaMA extended family also includes two fine-tuned models, one for Python and one for Instruct, the latter of which “has [been] fine-tuned to generate helpful and safe answers in natural language,” and therefore, Meta says, should be used when generating new code from natural language prompts. That is, it returns safer, more expected and perhaps less creative responses.
VentureBeat's mission is to be a digital town square for technical decision-makers to gain knowledge about transformative enterprise technology and transact. Discover our Briefings.