bitnet.js
A node.js implementation of the microsoft bitnet.cpp inference framework
overview
it is the Node.js implementation of Microsoft's bitnet.cpp inference framework. This repository facilitates real-time interaction between a Node.js frontend and the bitnet 1-bit LLM model using Socket.IO. The app allows users to send queries to the BitNet LLM (Large Language Model) and receive responses line by line via a web interface.
why it exists
the actual inference framework by microsoft does not provide a web interface to interact with the model. so i decided to build my own implementation using Node.js and Socket.IO.
features
- A simple frontend built with HTML and JavaScript to interact with the BitNet model
- Uses Socket.IO for bi-directional communication between the Node.js app and the Python-based BitNet model
- Both the Node.js app and BitNet model run in separate Docker containers managed by docker compose
quick start
docker compose up --build -dstack
- html
- python
- dockerfile
- javascript
links
license
distributed under mit.
contribute
raise an issue or submit a pull request if you find any bug or want to add a feature. i will get some time to review and fix it or if its a pull request then i will merge it.