Llama 3.1

The open-source standard for general-purpose AI, scaling from 8B to 405B parameters.

Generic Info

Publisher: Meta AI
Release Date: July 2024
Parameters: 8B, 70B, 405B
Context Window: 128k tokens
License: Llama 3.1 Community License
Key Capabilities: Multilingual, Reasoning, Coding, Tool Use

Llama 3.1 represents a significant leap in open-source AI, offering performance comparable to top-tier closed models like GPT-4o. The 405B model is the largest open-weights model to date, enabling synthetic data generation and model distillation.

Hello World Guide

Get started with Llama 3.1 using the Hugging Face transformers library.

Python

import torch
from transformers import pipeline

model_id = "meta-llama/Meta-Llama-3.1-8B-Instruct"

pipe = pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device_map="auto",
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello! Can you explain quantum computing in one sentence?"},
]

outputs = pipe(
    messages,
    max_new_tokens=256,
)

print(outputs[0]["generated_text"][-1])

Industry Usage

Zoom

Uses Llama models to power its AI Companion, summarizing meetings and drafting emails for millions of users.

Shopify

Integrates Llama into its sidekick assistant to help merchants manage their stores and answer queries.

AT&T

Fine-tunes Llama for customer support, improving response times and accuracy in handling client requests.