Exploring PrivateGPT

2023-07-07T17:17:04+00:00

Started a 8GB debian droplet on Digital Ocean
sudo apt install git
git clone git@github.com:imartinez/privateGPT.git
cd privateGPT/
sudo apt install python3-pip
sudo apt install python3.11-venv
python3 -m venv privategptenv
source privategptenv/bin/activate
pip3 install -r requirements.txt
cp example.env .env
Fetched the model wget https://gpt4all.io/models/ggml-gpt4all-j-v1.3-groovy.bin
I had written crawler for the target website and saved output of each url in a text file.
mv *.txt privateGPT/source_documents/
pip install sentence_transformers
I had to use the fix for privateGPT.py from here https://github.com/imartinez/privateGPT/pull/881/files
python ingest.py
python privateGPT.py

For each query it took 5 to 6 minutes. Some queries were answered ok. But other responses were hallucination; the data of the response was not in the source documents.