Knowledge Server, version 3

Access controlled question answering over an organisation's documents. This machine is provisioned and serving over HTTPS. The application is still being built.

Infrastructure

HTTPS certificateissued, auto renewing
Firewall22, 80, 443, default deny
SSH password authenticationdisabled
Brute force protectionactive
Model serverloopback only
Graphics cardNVIDIA A16, 8 GB

Measured on this machine

Reading a prompt694 tokens per second
Writing an answer10.1 tokens per second
Time to the first word1.1 seconds
Data files profiled181 MB in 2.2 seconds

Application

Access policy and decision enginebuilt
Accounts, sessions, second factorbuilt
Retrieval and knowledge basesbuilt
Document and dataset ingestionbuilt
Chat interface with streamingin progress
Corpus importnot started