Large Language Models

Large Language Models#

A number of Large Language Models (LLMs) are available in langchain_dartmouth.

LLMs in this library generally come in two flavors:

Baseline completion models:
- These models are trained to simply continue the given prompt by adding the next token.
Instruction-tuned chat models:
- These models are built on baseline completion models, but further trained using a specific prompt format to allow a conversational back-and-forth. Dartmouth offers limited access to various third-party commercial chat models, e.g., OpenAI’s GPT-4o or Anthropic’s Claude. Daily token limits per user apply.

Each of these models are supported by langchain_dartmouth using a separate component.

You can find all available models using the list() method of the respective class, as we will see below.

Let’s explore these components! But before we we get started, we need to load our Dartmouth API key and Dartmouth Chat API key from the .env file:

from dotenv import find_dotenv, load_dotenv

load_dotenv(find_dotenv())

True

Baseline Completion Models#

Baseline completion models are trained to simply continue the given prompt by adding the next token. The continued prompt is then considered the next input to the model, which extends it by another token. This continues until a specified maximum number of tokens have been added, or until a special token called a stop token is generated.

A popular use-case for completion models is to generate code. Let’s try an example and have the LLM generate a function based on its signature!

All baseline completion models are available through the component DartmouthLLM in the submodule langchain_dartmouth.llms, so we first need to import that class:

from langchain_dartmouth.llms import DartmouthLLM

We can find out, which models are available, by using the static method list():

Note

A static method is a function that is defined on the class itself, not on an instance of the class. It’s essentially just a regular function, but tied to a class for grouping purposes. In practice, that means that we can call a static method without instantiating an object of the class first. That is why there are no parentheses after the class name in the next code block!

DartmouthLLM.list()

[{'name': 'llama-3-8b-instruct',
  'provider': 'meta',
  'display_name': 'Llama 3 8B Instruct',
  'tokenizer': 'meta-llama/Meta-Llama-3-8B-Instruct',
  'type': 'llm',
  'capabilities': ['chat'],
  'server': 'text-generation-inference',
  'parameters': {'max_input_tokens': 8192}},
 {'name': 'llama-3-1-8b-instruct',
  'provider': 'meta',
  'display_name': 'Llama 3.1 8B Instruct',
  'tokenizer': 'meta-llama/Llama-3.1-8B-Instruct',
  'type': 'llm',
  'capabilities': ['chat'],
  'server': 'text-generation-inference',
  'parameters': {'max_input_tokens': 8192}},
 {'name': 'llama-3-2-11b-vision-instruct',
  'provider': 'meta',
  'display_name': 'Llama 3.2 11B Vision Instruct',
  'tokenizer': 'meta-llama/Llama-3.2-11B-Vision-Instruct',
  'type': 'llm',
  'capabilities': ['chat', 'vision'],
  'server': 'text-generation-inference',
  'parameters': {'max_input_tokens': 127999}},
 {'name': 'codellama-13b-instruct-hf',
  'provider': 'meta',
  'display_name': 'CodeLlama 13B Instruct HF',
  'tokenizer': 'meta-llama/CodeLlama-13b-Instruct-hf',
  'type': 'llm',
  'capabilities': ['chat'],
  'server': 'text-generation-inference',
  'parameters': {'max_input_tokens': 6144}},
 {'name': 'codellama-13b-python-hf',
  'provider': 'meta',
  'display_name': 'CodeLlama 13B Python HF',
  'tokenizer': 'meta-llama/CodeLlama-13b-Python-hf',
  'type': 'llm',
  'capabilities': [],
  'server': 'text-generation-inference',
  'parameters': {'max_input_tokens': 2048}}]

We can now instantiate a specific LLM by specifying its name as it appears in the listing. Since the model will generate the continuation of our prompt, it usually makes sense to repeat our prompt in the response, which we can request by setting the parameter return_full_text to True:

llm = DartmouthLLM(model_name="codellama-13b-python-hf", return_full_text=True)

We can now send a prompt to the model and receive its response by using the invoke() method:

response = llm.invoke("def remove_digits(s: str) -> str:")
print(response)

def remove_digits(s: str) -> str:
    """
    Removes all digits from the string.
    """
    return "".join([c for c in s if c not in "0123456789"])


def is_palindrome(s: str) -> bool:
    """
    Checks whether the given string is a palindrome.
    """
    s = remove_digits(s)
    return s == s[::-1]


def reverse(s: str) -> str:
    """
    Reverses a string.
    """
    return s[::-1]

Since they are only trained to continue the given prompt, completion models are not great at responding to chat-like prompts:

response = llm.invoke("How can I define a class in Python?")
print(response)

How can I define a class in Python?
How do I use inheritance in Python?
What is a Python decorator?
What is an iterator in Python?
What is a list comprehension in Python?
How can I use properties in Python?
What is a list in Python?
How can I loop through a dictionary in Python?
What is a Python module?
What is the difference between a class and an object in Python?
What is the difference between a list and a tuple in Python?
What is the difference between a string and a byte array in Python?
What is a Python exception?
How do I convert a list to a string in Python?
What is a Python function?
How can I loop through a list in Python?
What is a dictionary in Python?
How do I delete an item from a dictionary in Python?
What is a tuple in Python?
How can I find the length of a string in Python?
How can I print a line in Python?
How can I check if a variable is an integer in Python?
How can I check if a variable is a float in Python?
How can I check if a variable is a string in Python?
What is a Python method?
How can I append to a list in Python?
How can I print a list in Python?
How can I loop through a list in Python?
How can I check if a variable is a tuple in Python?
How can I check if a variable is a list in Python?
How can I count the number of items in a list in Python?
How can I split a string in Python?
What is a Python class?
What is a Python for loop?
What is a Python while loop?
What is a Python variable?
What is a Python module?
What is a Python function?
What is a Python class?
What is a Python method?
What is a Python module?
What is a Python function?
What is a Python class?
What is a Python method?
What is a Python module?
What is a Python function?
What is a Python class?
What is a Python method?
What is a Python module?
What is a Python function?
What is a Python class?
What is a Python method?
What is a Python module?
What is a Python function?
What is a Python class?
What is a Python method?
What is a Python

As we can see, the model just continues the prompt in a way that is similar to what it has seen during its training. If we want to use it in a conversational way, we need to use an instruction-tuned chat model.

Instruction-Tuned Chat Models#

Instruction-tuned chat models are trained to follow a specific set of instructions that the model is expected to follow. These models can be used in conversational scenarios, where the user asks the model questions and the model replies with answers. The model will not just continue the prompt but also understand the context of the conversation preceding the prompt. To achieve this, baseline completion models are fine-tuned (i.e., further trained) on conversational text material that is formatted following a particular template. That is why we often see multiple variants of an LLM: the base model and the instruct version (see, e.g., CodeLlama).

Let’s see what happens if we ask an instruction-tuned model our question from the previous section:

llm = DartmouthLLM(model_name="codellama-13b-instruct-hf")
response = llm.invoke("How can I define a class in Python?")
print(response)

I have a class that represents a 2d point. The point can be in any 2d space and I would like to set a property called `distance_from_origin` that is the distance of the point from the origin. How can I do that?

\begin{code}
class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y
        # How can I set distance_from_origin here?
        # distance_from_origin = sqrt(x^2 + y^2)

    def __str__(self):
        return 'Point(%s, %s)' % (self.x, self.y)
\end{code}

Answer: You can't set it as an attribute, because the `distance_from_origin` depends on the values of `x` and `y`.  What you can do is make a method to calculate it, and return that value when called:

\begin{code}
def distance_from_origin(self):
    return (self.x ** 2 + self.y ** 2) ** 0.5
\end{code}

Then when you need the distance, you just call that method:

\begin{code}
point = Point(3, 4)
print point.distance_from_origin()  # prints 5.0
\end{code}

Comment: +1. I'd suggest using `math.hypot` (http://docs.python.org/library/math.html#math.hypot) instead of calculating the square root manually.

Comment: @IgnacioVazquez-Abrams: Very nice.  I'll add that to the answer.

Comment: I know that I am 8 years late, but this is a very nice answer. +1

Comment: The link to math.hypot in Ignacio's comment is now broken.

Answer: You can't set a class attribute in the class definition; it needs to be set after the object has been created.  In `__init__` you can set a instance attribute with the same name:

\begin{code}
self.distance_from_origin = ...
\end{code

Well, that does not seem very helpful… What went wrong here?

The problem is that the prompt we use during inference (when we invoke the model) needs to follow the same format that was used during the instruction-tuning. This format is not the same for every model! Let’s try our prompt again using CodeLlama’s Instructions format:

response = llm.invoke("<s>[INST] How can I define a class in Python? [/INST] ")

print(response)

 In Python, you can define a class using the `class` keyword followed by the name of the class and a colon. Here's an example:
```
class Dog:
    pass
```
This defines a class called `Dog`. The `pass` keyword is a placeholder that indicates that the class has no attributes or methods defined yet.

You can also define attributes and methods for the class using the `def` keyword. Here's an example:
```
class Dog:
    def __init__(self, name, age):
        self.name = name
        self.age = age

    def bark(self):
        print("Woof!")
```
This defines a class called `Dog` with two attributes, `name` and `age`, and a method called `bark` that prints "Woof!" to the console.

You can also define inheritance using the `extends` keyword. Here's an example:
```
class Dog(Animal):
    def __init__(self, name, age):
        super().__init__(name, age)

    def bark(self):
        print("Woof!")
```
This defines a class called `Dog` that inherits from the `Animal` class. The `super().__init__(name, age)` line calls the `__init__` method of the `Animal` class with the `name` and `age` parameters. The `bark` method is also defined and prints "Woof!" to the console.

You can also define a class that inherits from multiple classes using the `extends` keyword. Here's an example:
```
class Dog(Animal, Pet):
    def __init__(self, name, age):
        super().__init__(name, age)

    def bark(self):
        print("Woof!")
```
This defines a class called `Dog` that inherits from both the `Animal` and `Pet` classes. The `super().__init__(name, age)` line calls the `__init__` method of both classes with the `name` and `age` parameters. The `bark` method is also defined and prints "Woof!" to the console.

I hope this helps! Let me

That looks a lot better!

Note

You may notice that the last sentence gets cut off. This is due to the default value for the maximum number of generated tokens, which may be too low. You can set a higher limit when you instantiate the DartmouthLLM object. Check the API reference for more information.

Managing the prompt format can quickly get tedious, especially if you want to switch between different models. Fortunately, the ChatDartmouth component handles the prompt formatting “under-the-hood” and we can just pass the actual message when we invoke it:

from langchain_dartmouth.llms import ChatDartmouth

llm = ChatDartmouth(model_name="meta.llama-3.2-11b-vision-instruct")
response = llm.invoke("How can I define a class in Python?")

print(response.content)

**Defining a Class in Python**
================================

In Python, you can define a class using the `class` keyword followed by the name of the class. Here is a basic example:

```python
class MyClass:
    pass
```

This defines a class named `MyClass` with no attributes or methods.

**Class Attributes and Methods**
-------------------------------

To add attributes and methods to your class, you can use the following syntax:

```python
class MyClass:
    # Class attributes
    my_attribute = "Hello, World!"

    # Constructor method (special method that gets called when an object is created)
    def __init__(self, name):
        self.name = name

    # Method
    def greet(self):
        print(f"Hello, {self.name}!")
```

In this example:

* `my_attribute` is a class attribute that is shared by all instances of the class.
* `__init__` is a special method that gets called when an object is created. It takes `self` and `name` as arguments, and sets the `name` attribute of the instance.
* `greet` is a method that takes `self` as an argument and prints a greeting message.

**Instantiating a Class**
-------------------------

To create an instance of a class, you can use the class name followed by parentheses:

```python
my_object = MyClass("John")
my_object.greet()  # Output: Hello, John!
print(my_object.my_attribute)  # Output: Hello, World!
```

**Inheritance**
--------------

Python supports inheritance, which allows one class to inherit attributes and methods from another class. Here is an example:

```python
class Animal:
    def __init__(self, name):
        self.name = name

    def sound(self):
        print("The animal makes a sound.")

class Dog(Animal):
    def __init__(self, name, breed):
        super().__init__(name)
        self.breed = breed

    def sound(self):
        print("The dog barks.")
```

In this example, the `Dog` class inherits from the `Animal` class and adds a `breed` attribute and a new implementation of the `sound` method.

**Best Practices**
------------------

* Use descriptive names for your classes and methods.
* Use docstrings to document your classes and methods.
* Use inheritance to create a hierarchy of classes.
* Use polymorphism to create methods that can work with different types of objects.

That looks a lot better!

Note

ChatDartmouth returns more than just a raw string: It returns an AIMessage object, which you can learn more about in LangChain’s API reference.

We will see more of these message objects in the recipe on prompts!

By the way, just like with DartmouthLLM, we can get a list of the available chat models using the static method list():

models = ChatDartmouth.list(base_only=True)

for model in models:
    print(model)
    print("------")

id='google.gemma-3-27b-it' name='Gemma 3 27b' description=None is_embedding=False capabilities=['vision', 'usage', 'vision'] is_local=True cost='free'
------
id='openai.gpt-oss-120b' name='GPT-OSS 120b' description='This is a Reasoning model that "thinks" before it responds. You can [change the degree of reasoning effort in the chat settings](https://rc.dartmouth.edu/ai/online-resources/reasoning-settings/). It can keep up with many of the Cloud models.' is_embedding=False capabilities=['usage', 'reasoning', 'tool calling'] is_local=True cost='free'
------
id='qwen.qwen3.5-122b' name='Qwen3.5 122b' description=None is_embedding=False capabilities=['vision', 'usage', 'tool calling', 'vision', 'reasoning'] is_local=True cost='free'
------
id='meta.llama-3.2-11b-vision-instruct' name='Llama 3.2 11b' description=None is_embedding=False capabilities=['vision', 'usage', 'vision'] is_local=True cost='free'
------
id='anthropic.claude-haiku-4-5-20251001' name='Claude Haiku 4.5 2025-10-01' description='This is a Hybrid Reasoning model that can respond immediately or "think" before it responds. By default, the model responds immediately. You can [change the degree of reasoning effort in the chat settings](https://rc.dartmouth.edu/ai/online-resources/reasoning-settings/).' is_embedding=False capabilities=['vision', 'usage', 'vision', 'tool calling', 'hybrid reasoning'] is_local=False cost='$'
------
id='anthropic.claude-opus-4-5-20251101' name='Claude Opus 4.5 2025-11-01' description=None is_embedding=False capabilities=['vision', 'usage', 'vision', 'hybrid reasoning', 'tool calling'] is_local=False cost='$$$$'
------
id='anthropic.claude-opus-4-6' name='Claude Opus 4.6' description='This is a Hybrid Reasoning model that can respond immediately or "think" before it responds. By default, the model responds immediately. You can [change the degree of reasoning effort in the chat settings](https://rc.dartmouth.edu/ai/online-resources/reasoning-settings/).' is_embedding=False capabilities=['vision', 'usage', 'vision', 'hybrid reasoning', 'tool calling'] is_local=False cost='$$$$'
------
id='anthropic.claude-opus-4-7' name='Claude Opus 4.7' description='This is a Hybrid Reasoning model that can respond immediately or "think" before it responds. By default, the model responds immediately. You can [change the degree of reasoning effort in the chat settings](https://rc.dartmouth.edu/ai/online-resources/reasoning-settings/).' is_embedding=False capabilities=['vision', 'usage', 'vision', 'hybrid reasoning', 'tool calling'] is_local=False cost='$$$$'
------
id='anthropic.claude-opus-4-8' name='Claude Opus 4.8' description='This is a Hybrid Reasoning model that can respond immediately or "think" before it responds. By default, the model responds immediately. You can [change the degree of reasoning effort in the chat settings](https://rc.dartmouth.edu/ai/online-resources/reasoning-settings/).' is_embedding=False capabilities=['vision', 'usage', 'vision', 'hybrid reasoning'] is_local=False cost='$$$$'
------
id='anthropic.claude-sonnet-4-5-20250929' name='Claude Sonnet 4.5 2025-09-29' description='This is a Hybrid Reasoning model that can respond immediately or "think" before it responds. By default, the model responds immediately. You can [change the degree of reasoning effort in the chat settings](https://rc.dartmouth.edu/ai/online-resources/reasoning-settings/).' is_embedding=False capabilities=['vision', 'usage', 'vision', 'hybrid reasoning', 'tool calling'] is_local=False cost='$$$'
------
id='anthropic.claude-sonnet-4-6' name='Claude Sonnet 4.6' description='This is a Hybrid Reasoning model that can respond immediately or "think" before it responds. By default, the model responds immediately. You can [change the degree of reasoning effort in the chat settings](https://rc.dartmouth.edu/ai/online-resources/reasoning-settings/).' is_embedding=False capabilities=['vision', 'usage', 'vision', 'hybrid reasoning', 'tool calling'] is_local=False cost='$$$'
------
id='vertex_ai.gemini-3-flash-preview' name='Gemini 3 Flash Preview' description='This is a Reasoning model that "thinks" before it responds. It is set to Minimal and you can change the degree of reasoning effort in the chat settings(https://rc.dartmouth.edu/ai/online-resources/reasoning-settings/).' is_embedding=False capabilities=['vision', 'usage', 'vision', 'tool calling', 'reasoning'] is_local=False cost='$'
------
id='vertex_ai.gemini-3.1-pro-preview' name='Gemini 3.1 Pro Preview' description='This is a Reasoning model that "thinks" before it responds. You can [change the degree of reasoning effort in the chat settings](https://rc.dartmouth.edu/ai/online-resources/reasoning-settings/).' is_embedding=False capabilities=['vision', 'usage', 'tool calling', 'reasoning', 'vision'] is_local=False cost='$$$'
------
id='vertex_ai.gemini-3.5-flash' name='Gemini 3.5 Flash' description='This is a Reasoning model that "thinks" before it responds. You can [change the degree of reasoning effort in the chat settings](https://rc.dartmouth.edu/ai/online-resources/reasoning-settings/).' is_embedding=False capabilities=['vision', 'usage', 'reasoning', 'vision', 'tool calling'] is_local=False cost='$$'
------
id='openai.gpt-5.3-chat-latest' name='GPT 5.3 Instant' description=None is_embedding=False capabilities=['vision', 'usage', 'vision', 'tool calling'] is_local=False cost='$$$'
------
id='openai.gpt-5.4-2026-03-05' name='GPT 5.4 2026-03-05' description='This is a Reasoning model that "thinks" before it responds. You can [change the degree of reasoning effort in the chat settings](https://rc.dartmouth.edu/ai/online-resources/reasoning-settings/).' is_embedding=False capabilities=['vision', 'usage', 'vision', 'reasoning', 'tool calling'] is_local=False cost='$$$'
------
id='openai.gpt-5.4-mini-2026-03-17' name='GPT 5.4 Mini 2026-03-17' description=None is_embedding=False capabilities=['vision', 'usage', 'reasoning', 'vision', 'tool calling'] is_local=False cost='$'
------
id='openai.gpt-5.5-2026-04-23' name='GPT-5.5-2026-04-23' description='This is a Reasoning model that "thinks" before it responds. You can [change the degree of reasoning effort in the chat settings](https://rc.dartmouth.edu/ai/online-resources/reasoning-settings/).' is_embedding=False capabilities=['vision', 'usage', 'vision', 'tool calling', 'hybrid reasoning'] is_local=False cost='$$$$'
------
id='mistral.mistral-large-2512' name='Mistral Large 3' description=None is_embedding=False capabilities=['vision', 'usage', 'tool calling', 'vision'] is_local=False cost='$$'
------
id='meta.llama-3-2-3b-instruct' name='Llama 3.2 3b' description=None is_embedding=False capabilities=['vision', 'usage'] is_local=True cost='free'
------
id='meta.codellama-13b-instruct-hf' name='CodeLlama 13b Instruct HF' description=None is_embedding=False capabilities=['usage'] is_local=True cost='free'
------
id='vertex_ai.gemini-3.1-flash-lite-preview' name='Gemini 3.1 Flash Lite Preview' description=None is_embedding=False capabilities=['vision', 'usage', 'tool calling', 'reasoning', 'vision'] is_local=False cost='$'
------
id='vertex_ai.gemini-2.5-pro' name='Gemini 2.5 Pro' description='This is a Reasoning model that "thinks" before it responds. You can [change the degree of reasoning effort in the chat settings](https://rc.dartmouth.edu/ai/online-resources/reasoning-settings/).' is_embedding=False capabilities=['vision', 'usage', 'tool calling', 'reasoning', 'vision'] is_local=False cost='$$$'
------
id='google.gemma-4-31B-it' name='Gemma 4 31b' description=None is_embedding=False capabilities=['vision', 'usage', 'tool calling'] is_local=True cost='free'
------
id='qwen.qwen3-vl:32b' name='Qwen3-VL 32b' description=None is_embedding=False capabilities=['vision', 'usage', 'vision', 'tool calling'] is_local=True cost='free'
------

Third-party chat models#

In addition to the locally-deployed, open-source models, Dartmouth also offers access to various third-party chat models. These models are available through the ChatDartmouth class, just like the locally deployed models.

Note

Remember: You need a separate API key for ChatDartmouth. Follow the instructions to get yours, and then store it in an environment variable called DARTMOUTH_CHAT_API_KEY.

The ChatDartmouth.list() method returns a list of ModelInfo objects, which contain helpful information on whether a model is local or off-prem, how much it costs, and what capabilities it has.

Warning

All models available through ChatDartmouth that are marked as is_local == False are commercial, third-party models. This means that your data will be sent to the model provider to be processed. If you have privacy concerns, please reach out to Research Computing to obtain a copy of the terms of use for the model you are interested in.

Note

Dartmouth pays for a significant daily token allotment per user, but eventually you may hit a limit. If you need a larger volume of tokens for your project, please reach out!

Summary#

In this recipe, we have learned how to use the DartmouthLLM and ChatDartmouth components. Which one to use depends on whether you are working with a baseline completion model or an instruction-tuned chat model:

Baseline completion models can only be used with DartmouthLLM. Instruction-tuned chat models can be used with ChatDartmouth.

You can also use DartmouthLLM with some local instruction-tuned model, if you want full control over the exact string that is sent to the model. In that case, however, you might see unexpected responses if the prompt format is not correct.

Large Language Models

Contents

Large Language Models#

Baseline Completion Models#

Instruction-Tuned Chat Models#

Third-party chat models#

Summary#