Install Ollama publicly on your Digital Ocean droplet and use it from anywhere!

Perhaps you’ve even launched a few applications or services leveraging OpenAI’s API. One of my early ventures into production-grade applications using this API was a Google Chrome Extension named OpenAI’s Study Buddy. My aim was to develop a tutor application that supports students worldwide in their educational pursuits. Based on my thesis, I believed that a personal, affordable tutor could significantly improve educational outcomes for students across various socio-economic backgrounds and geographical locations. While there are many AI tutor and educational platforms, my objective was to utilize AI to help students learn by offering hints rather than outright answers, thereby fostering a deeper understanding.

Ollama is an AI framework designed for building and running language models locally. It offers a straightforward API for creating, running, and managing models, along with a library of pre-built models that can be seamlessly integrated into various applications. Ollama also simplifies the process of deploying your own LLM on a virtual machine, such as a Digital Ocean Droplet.

To get started, you’ll need to set up a new Digital Ocean Droplet. It’s important to research the requirements of your chosen model before selecting your droplet. For instance, you can run the base llama2 model on an 8GB droplet, costing approximately $50 USD. Although this might seem costly compared to your Netlify or Vercel fees, heavy OpenAI API users will likely experience substantial savings within the first month.

After completing the necessary installation steps, you need to allow traffic to your service from IP addresses different from your Digital Ocean droplet. The steps for this are covered in the accompanying video. Once you’ve stopped your service, adjusted the traffic permissions, and restarted your service, the original curl request should work successfully!