Show HN: AI Runner – my personal opensource, local, multi-modal, AI assistant https://ift.tt/v4s3oel

Show HN: AI Runner – my personal opensource, local, multi-modal, AI assistant AI Runner is an application I have been working on for well over a year. I am the only developer on the project. I currently do not have a job so I have treated this as my full-time job. 3.0 is not yet ready for release, but I wanted to show you some of the cool things I've done with it (see video in link or check out my youtube page for past progress updates https://www.youtube.com/@capsizegames4331 ). AI Runner is a privacy-forward, multi-modal offline AI application built with Pyside6, huggingface libraries, llamaindex and open source models. It allows for real-time text to speech conversations with chatbot agents that you can fully customize. The bots have personalities and "moods". The moods shift with the conversation. I am using custom agents and my own system for selecting actions which is very fast. The application also allows you to generate images using Stable Diffusion and controlnet (previous versions allowed Kandinsky), LLM for chatbots and RAG retrieval using Mistral 7b and T5, multi-lingual text-to-speech using speech-t5, bark and espeak. The whole thing can run on a single 2080s video card, or you can split it between multiple GPUs by assigning models to available cards. For example, I have a 2080s which I use for Stable Diffusion, and a 3060 that I use for the other models. I've built in lots of features such as drawing tools, RAG for documents and websites, image generation with the LLM or custom prompts and more. I post about it often on my Twitter acount ( https://twitter.com/xw4ffl35x ). In addition to being a stand-alone application, the goal is to make it usable as an engine to power other applications. For example I plan to use it to power an RPG I am developing in Pygame. The application is pure python and can be compiled with pyinstaller using the installation scripts included in the main repo ( https://ift.tt/XlOY4Uv ). I am currently working on v3.0.0 which is under heavy development and is not stable. The last stable version came out over 8 months ago and was version 2.2 (you can find it tagged on github). This new version is a complete refactor and uses proper threading, has enhanced privacy and security features (the app is not able to access the internet and complete works around any security holes in the huggingface libraries by preventing them from accessing the internet or creating a .cache folder - there's more information about this in the README). --- I believe it is crucially important to have open source options when it comes to AI. I do not want to live in a world that is dominated by OpenAI. I want free, open source models and the ability to run them locally, offline for free. If you would like to support my efforts, please consider donating via patreon https://ift.tt/r9iTqAL https://twitter.com/xw4ffl35x/status/1778094355812700635 May 11, 2024 at 02:12AM