I’ll admit it: I feel uncomfortable about mixing open-source software with proprietary platforms. There’s no technical reason for my hesitancy. We do it all the time with tools like Google Analytics or by connecting a form to our mailing list provider. But the idea seems cringey for an advocate of the open web.
So while connecting, say, WordPress to an AI model like ChatGPT, Gemini, or Claude will work, there are also concerns. What can these models access? How will they use my website’s data? Could a security leak put my site at risk? And what will it cost?
There’s also a thought that these models may be overkill for what I need. If I only want AI to help me change settings or analyze SEO, do I really want the entirety of what ChatGPT offers? It seems a bit wasteful.
That’s why I’m excited about the potential of open-source AI models – particularly ones that you can install locally. Such apps could be a solution to privacy and security concerns. They may also be more efficient than the electricity-hogging corporate models.
Let’s take a moment to dream about the possible benefits for agencies, freelancers, and website owners. I’m not a software engineer, so I can’t claim to know all the technical requirements. So, consider this a wish list. Even so, it’s still worth exploring.
A Cheaper Way To Integrate With AI
It’s no secret that serious AI integrations cost money. We pay the big players to access their APIs via credits or tokens. The more you use AI, the more you pay. It may be well worth the cost in some situations.
But like any use-based pricing system, things can get out of hand. For instance, what happens if you forget to set a spending limit? A spammer could flood your chatbot with requests, leading to an unseemly bill at the end of the month. A traffic spike could also lead to running out of tokens in short order.
A free-to-use open-source model could bring more cost certainty. Granted, much depends on the app’s size and resource usage. But the right one could make it easier to predict your AI spending. It might be a matter of upgrading your web hosting.
Open-source could do for AI what WordPress did for content management systems (CMS). Imagine spinning up an AI instance whenever you want. Hosts might even enable one-click installs.
Best of all, this scenario would help us avoid regular price increases. That’s key for those working on a tight budget.
An AI Model Trained on What You Need
The big AI apps are trained on massive amounts of data. It’s akin to swallowing the entire internet. You might feel a bit bloated in the aftermath.
These large language models (LLMs) are great for a multitude of uses. It can help your child with math homework, help you write a proposal, or generate out-of-this-world images. Very cool! And if you want your website to do all of these things, more power to you.
However, I suspect that’s beyond what most people will need. Maybe you want a model trained only on your product’s documentation so users can ask questions. Or you want to translate your content into another language. Smaller, open-source models might be a better fit for your niche.
This would also solve the problem of users tricking AI into doing undesirable things. A model that doesn’t have such capabilities could be a safer choice.
Again, this also plays into better performance and lower resource usage. The hope is for an AI app that doesn’t do all the things; rather, it focuses on the things you need.
Resources
Smart people are already working on various open-source AI models. It’s fair to say none have overtaken Claude or ChatGPT yet. But perhaps they don’t need to. After all, my hope is for leaner and more focused models.
We’re still in the early stages, but some interesting options have appeared. With that in mind, here are a few resources to help you find and experiment with current models:
- Lists of open-source artificial intelligence software (Wikipedia)
- LocalLLM – A subreddit full of info on local AI models.
- Ollama – A tool for connecting with open models that works with Linux, MacOS, and Windows.
- OpenClaw – A personal AI agent app that can be extended with skills and plugins.
- Open Source AI – The Open Source Initiative provides definitions, white papers, and other helpful information.
New models and variants are being released all the time. The links above will help you keep track of what’s new and interesting.
Is Open Source the Way to a Better AI?
It’s no secret that there are privacy and safety concerns with AI, not to mention the fear that this technology will replace humans at scale. While still a drop in the overall bucket, open-source models could be a ray of hope.
The idea of data ownership is appealing, as is AI that focuses on specific tasks. Done right, an open-source model could make AI more personal and less scary. These are goals worth exploring.
Time will tell whether one or more models will succeed. In the meantime, I think we owe it to ourselves to test and provide feedback. Together, we can push for AI that benefits everyone. That’s the open-source spirit!
The post The Potential of Local Open-Source AI Models appeared first on Speckyboy Design Magazine.