Running a Local LLM (hybrid event)

Join us for this LUG event as we walk through the steps for running a local LLM within a container, discussing why you would want to run a local LLM, and how to configure Open Web UI to use your local LLMs.

Prerequisites for the Demo
If you’d like to follow along during the demo, please carefully read everything below. We won’t have time to bootstrap everyone’s computer during the event. Some familiarity with your operating system’s command line will be required as we’ll be using commands to get everything up and running. Using Linux is not a requirement as the commands should also work on Windows and MacOS (though there might be some slight variances).

Having a computer with a LLM chip or any other specific hardware is not required. Everything we’ll be doing can be run on the CPU. The presentation will cover some hardware considerations and limitations.

What you’ll need:

• Laptop (if in-person)
• If using Windows or MacOS, make sure Podman Desktop is installed and up to date
• If using Linux, make sure Podman and/or Podman Desktop is installed and up to date

If you have any questions about any of the prerequisites or need help, please join the Hackforge Discord and ask your questions in the #linux-users-group channel. Our user base will be more than happy to help you!

Agenda

• Brief presentation covering:
  • What we’ll be doing in the demo
  • Discuss a couple of reasons why you would want to run a local LLM
  • Hardware considerations & limitations
  • Demo

Location: Hackforge HQ (300 Ouellette Ave., Unit 210, Windsor, ON) & Zoom

DETAILS