What is Groq?

  • Groq (not Grok!) is a company that runs AI models super fast and specializes in inferencing.
  • Inferencing is the process of using a trained AI model to generate predictions or responses based on new input data.
  • Groq is a great way to experiment with AI chatbots because it offers a generous free tier and fast response times.

Create a Groq Account and API Key:

  1. Go to console.groq.com and sign up for a free account using your email, Google account, or GitHub account.
  2. Once logged in, find API Keys in the navigation menu.
  3. Click Create API Key, give it a name (like “cis240”), and click Submit.
  4. Groq will show you the key exactly once. Copy it and save it somewhere safe, such as a plain text file on your computer. If you lose it, you’ll need to generate a new one.
  5. Treat your API key like a password. Never share it publicly or post it in a place like GitHub, a public webpage, or a group chat. Anyone with your key can use your account and free usage limits.

How to Install Groq:

1pip install groq

Putting It All Together with Dictionaries and the Groq API:

An AI Chatbot:

  • Test your Groq API key by calling the Groq API to generate an output from a model. The list of dictionaries in the messages variable is used to provide context for the AI model.
python
1api_key = "Paste your Groq API key here!"
2from groq import Groq
3import os
4os.environ["GROQ_API_KEY"] = api_key
5
6# Create an instance of Groq
7groq = Groq()
8
9# `messages` is a list of dictionaries representing the prompt.
10messages = [
11    {
12        "role": "system",
13        "content": "You are a helpful assistant that speaks like Cardi B.",
14    },
15    {
16        "role": "user",
17        "content": "Describe Python's most popular use cases in two sentences.",
18    },
19]
20
21# Call the create fuction to generate a message
22completion = groq.chat.completions.create(
23    model = "openai/gpt-oss-20b", 
24    messages = messages, 
25    max_tokens = 300, 
26    temperature = 1
27)
28print(completion.choices[0].message.content)

Example Output:

1Python is poppin' for web development, data analysis, and machine learning.
  • In line 1, an API key is stored as a string. This key is used to authenticate the user and is required to access the Groq API.
  • In line 2, the Groq package is imported. This package is used to interact with the Groq API.
  • In lines 3 and 4, the API key is set as an environment variable. Environment variables can store information such as file paths, system settings, and passwords.
  • In line 7, an instance of the ‘Groq’ class is created and stored in the variable ‘groq’.
  • In lines 10-19, a list named, messages is defined, containing dictionaires that represent the prompt for the AI. Each dictionary specifies the roles (‘system’ or ‘user’) and the content of the message. The system’s role is to provide instructions, and the user’s role is to ask a question or make a statement.
  • In lines 22-27, the groq.chat.completions.create function is called to generate a text completion from the Groq API. This function takes several arguments:
    • model: Specifies the AI model to use, in this case, “openai/gpt-oss-20b”.
    • messages: Passes the conversation context defined earlier.
    • max_tokens: The maximum number of tokens (pieces of text) to generate. Set to 300 here.
    • temperature: Controls the randomness of the output. A value of 0 means the output will be deterministic and based on the most likely response.
  • Line 28: The generated text is extracted from the response object. The response object includes multiple fields, and choices is a Python list of possible completions. This script selects the first completion (choices[0]) and then accesses its message.content. The extracted text is printed to the terminal.