Large Language Models and Applications

Fall 26

Calvin Williamson, Science and Math

Large Language Models (LLM)

  • AI Models You Send a Prompt and They Give a Response
  • ChatGPT, Claude, Gemini, DeepSeek, Many Others
  • Prompts can include Text, Images, Audio, Video
  • Responses can include Text, Images, Audio, Video

Text Based LLMs

To prompt an LLM with text you type your question and submit it

sequenceDiagram
    participant User
    participant LLM
    User->>LLM: Prompt: What is the capital of France?
    LLM->>User: Response: The capital of France is Paris.

The model responds with some text.

Example: ChatGPT 3.5 (Nov 22)

MultiModal LLMs

Prompts can include

  • Text
  • Image (eg. uploaded image file)
  • Audio (eg. from microphone, or uploaded audio file)
  • Video (eg Webcam)
  • Screen sharing (eg show your computer screen)

Responses can include: Text, Image, Audio, Video

Example: Gemini 2.0 Flash (Dec 24)

How To Prompt a MultiModal LLM using Audio

To prompt the LLM you speak into a microphone on your computer

sequenceDiagram
    participant User
    participant LLM
    User->>LLM: Audio Prompt 🔊: Can you hear me? 
    LLM->>User: Audio Response 🔊: Yes! I hear you? How can I help you today?

The model responds in audio and/or text that you can hear and see immediately

How To Prompt a MultiModal LLM using Video and Audio

Using a webcam you speak and show the model what you want

sequenceDiagram
    participant User
    participant LLM
    User->>LLM: Webcam Prompt 📹+🔊: Can you see and hear me? 
    LLM->>User: Audio Response 🔊: Yes! I can both see and hear you. How can I help you today?

The model responds in audio and/or text

Large Language Model: Deep Seek

In Spring 25 a new model called Deep Seek caused Wall Street to panic and lose Billions of dollars. The reason is the Deep Seek was released as an open source model that anyone could download and run on their own, thus challenging the idea that only companies with proprietary models would dominate the AI space. The models from OpenAI, Anthropic, Google all are proprietary models:

  • ChatGPT (OpenAI)
  • Claude (Anthropic)
  • Gemini (Google)

These models cannot be run by anyone the way Deep Seek can.

Examples of Reasoning Models: Deep Seek

  1. Barbie Deep Think
  2. Making a Todo App with DeepSeek | Version 1 | Version 2

Deep Seeks models are competive with the state of the art closed source models but are created at a fraction of the cost.

Notice the difference when posting prompts to the standard V3 models versus the “reasoning model” (Button “DeepThink”). The reasoning model shows you its thinking process.

Large Language Models as Software Developers

  • Language models excel at writing software
  • Vibe Coding: Prompting models to write code without reading source
    • GPT-5.6 Sol (OpenAI)
    • Claude Opus 5 and Sonnet 5 (Anthropic)
    • Gemini 3.1 Pro (Google)
    • Qwen3-Coder-Next (Alibaba)
    • Kimi K3 (Moonshot)
    • GLM-5.2 (Z.ai)
    • DeepSeek V4 (DeepSeek)

Prompting Large Language Models

Some examples of prompting and their costs using openrouter:

Prompt Editor

LLMs and Tools

LLMs can call Tools (external programs)

This ability is sometimes called agentic.

Examples (Tools):

  • Websearch
  • Calculator
  • Wikipedia access
  • Financial Data APIs (Yahoo Finance)

Example LLM with a simple tool

A Modern AI Application

On the following slides we show a web application with no AI, and then show how it turns into a modern AI application.

The Original (non-AI) App

This is the FIT Retail Index (non-AI). It uses financial data for retail companies and lets you compare one company against another, or against the segment it belongs to. There is no AI anywhere in this version.

  • This app is hosted freely on Github. It does not need a traditional server to host it.
  • The app contacts a DoltHub database through an standard REST API that holds all of that financial data, and the web app reads the companies and their numbers directly from it.

The AI-version

This is the FIT Retail Index (AI version) which now has LLM Chat added.

Two things are new:

  • The chat can reach the same DoltHub database the original app uses, so the LLM can query the company data itself.
  • With some tools, the AI can also see the state of the menus on the company to company page. It can set those menus for you.

The AI can drive the user interface and access the same data.

A Music Composition Language

Strudel.cc is a musical language for live coding and DJing It is based on a language called Tidal Cycles which is written in a programming language called Haskell. It is compact, technical, and mathematical language.

It is amazing in capability.

Learning Strudel

Large Language Models Working With Strudel

Examples:

Examples of what the LLM knows

Call and Response

Examples of what the LLM knows

Chords And Melodies

Large Language Models Working With DAWs

Claude working with Patterns/Channels:

Large Language Models Working With DAWs

More Claude.ai to FL Studio (Mac):

Large Language Models Working With DAWs

DeepSeek working with a DAW: