Large Language Models

Large Language Models#

A number of Large Language Models (LLMs) are available in langchain_dartmouth.

LLMs in this library generally come in three flavors:

Baseline completion models:
- These models are trained to simply continue the given prompt by adding the next token.
Instruction-tuned chat models:
- These models are built on baseline completion models, but further trained using a specific prompt format to allow a conversational back-and-forth.
Commercial third-party chat models:
- Dartmouth offers limited access to various third-party commercial models, e.g., OpenAI’s GPT-4o or Anthropic’s Claude. Daily token limits per user apply.

Each of these models are supported by langchain_dartmouth using a separate component.

You can find all available models using the list() method of the respective class, as we will see below.

Let’s explore these components! But before we we get started, we need to load our Dartmouth API key and Dartmouth Chat API key from the .env file:

from dotenv import find_dotenv, load_dotenv

load_dotenv(find_dotenv())

True

Baseline Completion Models#

Baseline completion models are trained to simply continue the given prompt by adding the next token. The continued prompt is then considered the next input to the model, which extends it by another token. This continues until a specified maximum number of tokens have been added, or until a special token called a stop token is generated.

A popular use-case for completion models is to generate code. Let’s try an example and have the LLM generate a function based on its signature!

All baseline completion models are available through the component DartmouthLLM in the submodule langchain_dartmouth.llms, so we first need to import that class:

from langchain_dartmouth.llms import DartmouthLLM

We can find out, which models are available, by using the static method list():

Note

A static method is a function that is defined on the class itself, not on an instance of the class. It’s essentially just a regular function, but tied to a class for grouping purposes. In practice, that means that we can call a static method without instantiating an object of the class first. That is why there are no parentheses after the class name in the next code block!

DartmouthLLM.list()

[{'name': 'llama-3-8b-instruct',
  'provider': 'meta',
  'display_name': 'Llama 3 8B Instruct',
  'tokenizer': 'meta-llama/Meta-Llama-3-8B-Instruct',
  'type': 'llm',
  'capabilities': ['chat'],
  'server': 'text-generation-inference',
  'parameters': {'max_input_tokens': 8192}},
 {'name': 'llama-3-1-8b-instruct',
  'provider': 'meta',
  'display_name': 'Llama 3.1 8B Instruct',
  'tokenizer': 'meta-llama/Llama-3.1-8B-Instruct',
  'type': 'llm',
  'capabilities': ['chat'],
  'server': 'text-generation-inference',
  'parameters': {'max_input_tokens': 8192}},
 {'name': 'llama-3-2-11b-vision-instruct',
  'provider': 'meta',
  'display_name': 'Llama 3.2 11B Vision Instruct',
  'tokenizer': 'meta-llama/Llama-3.2-11B-Vision-Instruct',
  'type': 'llm',
  'capabilities': ['chat', 'vision'],
  'server': 'text-generation-inference',
  'parameters': {'max_input_tokens': 127999}},
 {'name': 'codellama-13b-instruct-hf',
  'provider': 'meta',
  'display_name': 'CodeLlama 13B Instruct HF',
  'tokenizer': 'meta-llama/CodeLlama-13b-Instruct-hf',
  'type': 'llm',
  'capabilities': ['chat'],
  'server': 'text-generation-inference',
  'parameters': {'max_input_tokens': 6144}},
 {'name': 'codellama-13b-python-hf',
  'provider': 'meta',
  'display_name': 'CodeLlama 13B Python HF',
  'tokenizer': 'meta-llama/CodeLlama-13b-Python-hf',
  'type': 'llm',
  'capabilities': [],
  'server': 'text-generation-inference',
  'parameters': {'max_input_tokens': 2048}}]

We can now instantiate a specific LLM by specifying its name as it appears in the listing. Since the model will generate the continuation of our prompt, it usually makes sense to repeat our prompt in the response, which we can request by setting the parameter return_full_text to True:

llm = DartmouthLLM(model_name="codellama-13b-python-hf", return_full_text=True)

We can now send a prompt to the model and receive its response by using the invoke() method:

response = llm.invoke("def remove_digits(s: str) -> str:")
print(response)

def remove_digits(s: str) -> str:
    i = 0
    while i < len(s):
        if s[i].isdigit():
            s = s[:i] + s[i + 1 :]
            if i:
                i -= 1
        else:
            i += 1
    return s

Since they are only trained to continue the given prompt, completion models are not great at responding to chat-like prompts:

response = llm.invoke("How can I define a class in Python?")
print(response)

How can I define a class in Python?
What is a class in Python?
What is inheritance in Python?
What is encapsulation in Python?
What is the difference between a class and an object in Python?
What is the difference between a method and a function in Python?
What is the difference between an instance and an object in Python?
How can I call a method of an object?
How can I access a property of an object?
How can I access a static variable of a class?
What is the use of a class in Python?
What is the use of a class in Python? Classes are the building blocks of object-oriented programming. Python is an object-oriented programming language. It means that everything in Python is an object, and everything has attributes and methods.
The class is the blueprint for an object. It is the prototype of a future object.
To create a class in Python, use the class keyword.
To define a class in Python, you need to specify the name of the class, and the methods and attributes of the class.
To access the class, you have to use the keyword class followed by the name of the class.
To create a class in Python, you have to define a class in Python, and to access the class, you have to use the keyword class.
Inheritance is the ability to create a class by inheriting attributes and methods of another class.
The base class is the class that is inherited from.
The derived class is the class that inherits attributes and methods from a base class.
To inherit a class in Python, use the keyword class followed by the name of the derived class, the keyword inherit, and the name of the base class.
To access a base class in Python, you have to use the keyword class followed by the name of the derived class.
To access a base class in Python, you have to use the keyword inherit.
How to access the base class in Python?
To access the base class in Python, you have to use the keyword inherit.
To create an object of a class in Python, use the keyword class followed by the name of the object and the name of the class.
How can I call a method of an object in Python?
How can I access a property of an object in Python?
How can I access a static variable of a class in Python?
What is the difference between a method and a function in Python?
A

As we can see, the model just continues the prompt in a way that is similar to what it has seen during its training. If we want to use it in a conversational way, we need to use an instruction-tuned chat model.

Instruction-Tuned Chat Models#

Instruction-tuned chat models are trained to follow a specific set of instructions that the model is expected to follow. These models can be used in conversational scenarios, where the user asks the model questions and the model replies with answers. The model will not just continue the prompt but also understand the context of the conversation preceding the prompt. To achieve this, baseline completion models are fine-tuned (i.e., further trained) on conversational text material that is formatted following a particular template. That is why we often see multiple variants of an LLM: the base model and the instruct version (see, e.g., CodeLlama).

Let’s see what happens if we ask an instruction-tuned model our question from the previous section:

llm = DartmouthLLM(model_name="codellama-13b-instruct-hf")
response = llm.invoke("How can I define a class in Python?")
print(response)

\begin{code}
class People:
    def __init__(self):
        self.name = ""
        self.age = 0

    def __str__(self):
        return "Name : " + self.name + ", Age : " + self.age
\end{code}

What is the problem with this?

Comment: What's the problem?

Answer: There is nothing wrong with this code. It is a class definition. The class `People` has two methods:
\begin{itemize}
\item `__init__` is the constructor, it initializes the object when it is created. It takes an optional argument `name`, which will be used to initialize the object's `self.name` attribute. If `name` is not given, it will be set to an empty string.
\item `__str__` is a method that is used to get a string representation of an object. This is what happens when you call `str()` on an object, or when you try to print it. It returns a string that looks like `Name : [object name], Age : [object age]`.
\end{itemize}

For example:

\begin{code}
p1 = People("John")
p2 = People()
print(p1)
print(p2)
\end{code}

Output:

\begin{code}
Name : John, Age : 0
Name : , Age : 0
\end{code}

Note that there is no `age` attribute set on the object `p2`, so it prints `Age : 0`.

Comment: how can i call the class ?

Comment: You create an object of the class. `p = People()` creates a new object of type `People`, and assigns it to the variable `p`.

Comment: How can I pass the parameter in the class?

Comment: @ShabbirAhmed That's what `__init__` is for. It is a constructor, and it takes a parameter called `name`.

Comment: I'm getting the error while calling the class:TypeError: __init__() takes 1 positional argument but 2 were given

Comment: @ShabbirAhmed You need to pass `name` to the constructor.

Well, that does not seem very helpful… What went wrong here?

The problem is that the prompt we use during inference (when we invoke the model) needs to follow the same format that was used during the instruction-tuning. This format is not the same for every model! Let’s try our prompt again using CodeLlama’s Instructions format:

response = llm.invoke("<s>[INST] How can I define a class in Python? [/INST] ")

print(response)

 In Python, you can define a class using the `class` keyword followed by the name of the class and a colon. Here is an example:
```
class MyClass:
    pass
```
This defines a class called `MyClass` with no methods or variables.

You can also define a class with a constructor (also known as an initializer) using the `__init__` method. This method is called when an instance of the class is created. Here is an example:
```
class MyClass:
    def __init__(self, x):
        self.x = x
```
This defines a class called `MyClass` with a constructor that takes an argument `x`. The constructor sets the value of the `x` attribute of the class instance to the value of the argument `x`.

You can also define methods within a class using the `def` keyword. Here is an example:
```
class MyClass:
    def __init__(self, x):
        self.x = x

    def my_method(self):
        print("Hello, world!")
```
This defines a class called `MyClass` with a constructor and a method called `my_method`. The `my_method` method prints the string "Hello, world!" to the console.

You can also define properties using the `@property` decorator. Here is an example:
```
class MyClass:
    def __init__(self, x):
        self.x = x

    @property
    def my_property(self):
        return self.x * 2
```
This defines a class called `MyClass` with a constructor and a property called `my_property`. The `my_property` property returns the value of the `x` attribute of the class instance multiplied by 2.

You can also define classes that inherit from other classes using the `super()` function. Here is an example:
```
class MyClass:
    def __init__(self, x):
        self.x = x

class MyOtherClass(MyClass):
    def __init__(self, x):
        super().__init__(x)
```
This defines a class called `MyOtherClass` that inherits from the `MyClass` class. The `MyOtherClass` class has a constructor that calls the constructor of the `My

That looks a lot better!

Note

You may notice that the last sentence gets cut off. This is due to the default value for the maximum number of generated tokens, which may be too low. You can set a higher limit when you instantiate the DartmouthLLM object. Check the API reference for more information.

Managing the prompt format can quickly get tedious, especially if you want to switch between different models. Fortunately, the ChatDartmouth component handles the prompt formatting “under-the-hood” and we can just pass the actual message when we invoke it:

from langchain_dartmouth.llms import ChatDartmouth

llm = ChatDartmouth(model_name="llama-3-1-8b-instruct")
response = llm.invoke("How can I define a class in Python?")

print(response.content)

**Defining a Class in Python**
=====================================

In Python, you can define a class using the `class` keyword followed by the name of the class and a colon. The class definition is contained within a block of code, which is indented.

**Basic Syntax**
---------------

```python
class ClassName:
    # class body
```

**Example**
-----------

Here's an example of a simple `Person` class:

```python
class Person:
    def __init__(self, name, age):
        self.name = name
        self.age = age

    def greet(self):
        print(f"Hello, my name is {self.name} and I am {self.age} years old.")
```

**Explanation**
--------------

*   `class Person:` defines a new class named `Person`.
*   `def __init__(self, name, age):` defines a special method called the constructor, which is called when an instance of the class is created. It initializes the `name` and `age` attributes.
*   `self.name = name` and `self.age = age` assign the `name` and `age` arguments to instance variables.
*   `def greet(self):` defines a method called `greet` that prints a message to the console.

**Creating an Instance**
----------------------

You can create an instance of the `Person` class like this:

```python
person = Person("John", 30)
person.greet()
```

This will output:

```
Hello, my name is John and I am 30 years old.
```

**Attributes and Methods**
-------------------------

Classes can have attributes (data) and methods (functions). Attributes are variables that are defined inside a class, and methods are functions that belong to a class.

**Attributes**
-------------

Attributes can be accessed using the dot notation, like this:

```python
person.name
person.age
```

**Methods**
------------

Methods are called using the dot notation, like this:

```python
person.greet()
```

**Inheritance**
--------------

Python supports inheritance, which allows one class to inherit the attributes and methods of another class.

```python
class Employee(Person):
    def __init__(self, name, age, department):
        super().__init__(name, age)
        self.department = department

    def greet(self):
        super().greet()
        print(f"I work in the {self.department} department.")
```

In this example, the `Employee` class inherits

That looks a lot better!

Note

ChatDartmouth returns more than just a raw string: It returns an AIMessage object, which you can learn more about in LangChain’s API reference.

We will see more of these message objects in the recipe on prompts!

By the way, just like with DartmouthLLM, we can get a list of the available chat models using the static method list():

ChatDartmouth.list()

[{'name': 'llama-3-8b-instruct',
  'provider': 'meta',
  'display_name': 'Llama 3 8B Instruct',
  'tokenizer': 'meta-llama/Meta-Llama-3-8B-Instruct',
  'type': 'llm',
  'capabilities': ['chat'],
  'server': 'text-generation-inference',
  'parameters': {'max_input_tokens': 8192}},
 {'name': 'llama-3-1-8b-instruct',
  'provider': 'meta',
  'display_name': 'Llama 3.1 8B Instruct',
  'tokenizer': 'meta-llama/Llama-3.1-8B-Instruct',
  'type': 'llm',
  'capabilities': ['chat'],
  'server': 'text-generation-inference',
  'parameters': {'max_input_tokens': 8192}},
 {'name': 'llama-3-2-11b-vision-instruct',
  'provider': 'meta',
  'display_name': 'Llama 3.2 11B Vision Instruct',
  'tokenizer': 'meta-llama/Llama-3.2-11B-Vision-Instruct',
  'type': 'llm',
  'capabilities': ['chat', 'vision'],
  'server': 'text-generation-inference',
  'parameters': {'max_input_tokens': 127999}},
 {'name': 'codellama-13b-instruct-hf',
  'provider': 'meta',
  'display_name': 'CodeLlama 13B Instruct HF',
  'tokenizer': 'meta-llama/CodeLlama-13b-Instruct-hf',
  'type': 'llm',
  'capabilities': ['chat'],
  'server': 'text-generation-inference',
  'parameters': {'max_input_tokens': 6144}}]

Third-party chat models#

In addition to the locally-deployed, open-source models, Dartmouth also offers access to various third-party chat models. These models are available through the ChatDartmouthCloud class.

Note

Remember: You need a separate API key for ChatDartmouthCloud. Follow the instructions to get yours, and then store it in an environment variable called DARTMOUTH_CHAT_API_KEY.

from langchain_dartmouth.llms import ChatDartmouthCloud

As with the other classes, we can list the available models using the static method list():

ChatDartmouthCloud.list()

[{'name': 'anthropic.claude-3-5-haiku-20241022',
  'provider': 'anthropic',
  'type': 'llm',
  'capabilities': ['chat'],
  'server': 'dartmouth-chat',
  'parameters': {}},
 {'name': 'anthropic.claude-3-7-sonnet-20250219',
  'provider': 'anthropic',
  'type': 'llm',
  'capabilities': ['chat', 'vision'],
  'server': 'dartmouth-chat',
  'parameters': {}},
 {'name': 'google_genai.gemini-2.0-flash-001',
  'provider': 'google_genai',
  'type': 'llm',
  'capabilities': ['chat', 'vision'],
  'server': 'dartmouth-chat',
  'parameters': {}},
 {'name': 'google_genai.gemini-1.5-pro-002',
  'provider': 'google_genai',
  'type': 'llm',
  'capabilities': ['chat', 'vision'],
  'server': 'dartmouth-chat',
  'parameters': {}},
 {'name': 'mistral.pixtral-large-2411',
  'provider': 'mistral',
  'type': 'llm',
  'capabilities': ['chat', 'vision'],
  'server': 'dartmouth-chat',
  'parameters': {}},
 {'name': 'mistral.mistral-small-2503',
  'provider': 'mistral',
  'type': 'llm',
  'capabilities': ['chat', 'vision'],
  'server': 'dartmouth-chat',
  'parameters': {}},
 {'name': 'openai.gpt-4.1-2025-04-14',
  'provider': 'openai',
  'type': 'llm',
  'capabilities': ['chat', 'vision'],
  'server': 'dartmouth-chat',
  'parameters': {}},
 {'name': 'openai.gpt-4.1-mini-2025-04-14',
  'provider': 'openai',
  'type': 'llm',
  'capabilities': ['chat', 'vision'],
  'server': 'dartmouth-chat',
  'parameters': {}},
 {'name': 'openai.o4-mini-2025-04-16',
  'provider': 'openai',
  'type': 'llm',
  'capabilities': ['chat', 'vision'],
  'server': 'dartmouth-chat',
  'parameters': {}},
 {'name': 'meta-llama/Llama-3.2-3B-Instruct',
  'provider': 'meta-llama/Llama-3',
  'type': 'llm',
  'capabilities': ['chat'],
  'server': 'dartmouth-chat',
  'parameters': {}}]

Using the class works just like with the other two varieties:

llm = ChatDartmouthCloud(model_name="openai.gpt-4.1-mini-2025-04-14")

response = llm.invoke("Who are you?")

print(response.content)

I'm ChatGPT, an AI language model created by OpenAI. I'm here to help answer your questions, provide information, and assist with a wide range of topics. How can I assist you today?

Warning

The models available through ChatDartmouthCloud are commercial, third-party models. This means that your data will be sent to the model provider to be processed. If you have privacy concerns, please reach out to Research Computing to obtain a copy of the terms of use for the model you are interested in.

Note

Dartmouth pays for a significant daily token allotment per user, but eventually you may hit a limit. If you need a larger volume of tokens for your project, please reach out!

Summary#

In this recipe, we have learned how to use the DartmouthLLM, ChatDartmouth, ChatDartmouthCloud components. Which one to use depends on whether you are working with a baseline completion model, a local instruction-tuned chat model, or a Cloud-hosted third-party chat model:

Baseline completion models can only be used with DartmouthLLM. Local instruction-tuned chat models should be used with ChatDartmouth so the correct prompt format is applied automatically. For commercial third-party chat models, use ChatDartmouthCloud.

You can also use DartmouthLLM with an instruction-tuned model, if you want full control over the exact string that is sent to the model. In that case, however, you might see unexpected responses if the prompt format is not correct.

Large Language Models

Contents

Large Language Models#

Baseline Completion Models#

Instruction-Tuned Chat Models#

Third-party chat models#

Summary#