System Requirements

If users do not plan to run LLMs locally, then running Ailice has virtually no hardware requirements. For users who want to run LLMs locally, currently only models with 70B or more parameters can perform tasks well, therefore you need at least two RTX 4090 GPUs (48GB VRAM) to effectively complete tasks.

• Ailice was developed in Ubuntu, therefore installation and usage on Ubuntu has the best guarantee.
• The usability in a MacOS environment is similar to that in an Ubuntu environment.
• For Windows users, using Docker or installing WSL (Windows Subsystem for Linux) and running Ailice within WSL is a better choice, especially for users who need to execute programming tasks -- I haven't integrated Windows command execution tools yet (this will be considered in the future, but the flexibility of command-line tools is of great significance for AI agents, which makes Linux platforms more advantageous). Additionally, the lack of testing on Windows significantly increases the likelihood of bugs; if you encounter related issues, please submit issues for resolution.

Before installing Ailice, it is strongly recommended to install Anaconda and create a virtual environment first (you can also use other tools you prefer, such as venv). You will also need Chrome, as Ailice needs it for web browsing. For users who want to run Ailice in a fully controlled container/virtual machine, you will need Docker (or other virtual machines, such as VirtualBox).

If you want to run Ailice in a virtual machine, ensure Hyper-V is turned off(otherwise llama.cpp cannot be installed). In a VirtualBox environment, you can disable it by following these steps: disable PAE/NX and VT-X/AMD-V ( Hyper-V) on VirtualBox settings for the VM. Set paravirtualization Interface to Default, disable nested paging.

<a name="environment-configuration-and-installation"></a>

Quick Installation

Install and run Ailice with the following commands. Once Ailice is launched, use a browser to open the web page it provides, a dialogue interface will appear. Issue commands to Ailice through the conversation to accomplish various tasks. For your first use, you can try the commands provided in the COOL things we can do section to quickly get familiarized.

Local run:

git clone https://github.com/myshell-ai/AIlice.git
cd AIlice
pip install -e .
ailice --contextWindowRatio=0.2

Sandbox run:

git clone https://github.com/myshell-ai/AIlice.git
cd AIlice
docker build -t ailice .
docker run -it -p 127.0.0.1:5000:5000 --name ailice ailice --expose=1 --contextWindowRatio=0.2

Sandbox run with CUDA support(Please install nvidia-container-toolkit first):

git clone https://github.com/myshell-ai/AIlice.git
cd AIlice
docker build --build-arg BASE_IMAGE=nvidia/cuda:13.0.0-cudnn-devel-ubuntu24.04 -t ailice .
docker run --gpus all -it -p 127.0.0.1:5000:5000 --name ailice ailice --expose=1 --contextWindowRatio=0.2

Sandbox run with GUI support(Linux only, special configuration required for Windows and macOS):

git clone https://github.com/myshell-ai/AIlice.git
cd AIlice
docker build -t ailice .
docker run -it -p 127.0.0.1:5000:5000 \
    -e DISPLAY=$DISPLAY \
    -v /tmp/.X11-unix:/tmp/.X11-unix \
    --name ailice \
    ailice --expose=1 --contextWindowRatio=0.2

• For a more detailed understanding of the installation and configuration methods, please visit the Installation and Usage section and the Selection and Configuration of LLM section.
• To grasp the basic design principles of Ailice, navigate to the Design section.

<a name="cool-things-we-can-do"></a>

Installation and Usage

<a name="system-requirements"></a>

Environment Configuration and Installation

You can use the following command to install Ailice:

git clone https://github.com/myshell-ai/AIlice.git
cd AIlice
pip install -e .

The running speed of Ailice may be slow after installation, as the long-term memory module's embedded model computation might be running on CPU. In this case, we can try running the following command to install GPU acceleration:

ailice_turbo

For users who need to use the pdf reading/voice dialogue/huggingface models/model fine-tuning functions, you can use one of the following command(Installing too many features increases the likelihood of dependency conflicts, so it is recommended to install only the necessary parts):

pip install -e .[pdf-reading]
pip install -e .[speech]
pip install -e .[huggingface]
pip install -e .[finetuning]

You can run Ailice now! Use the commands in Usage.

<a name="if-you-need-to-frequently-use-google"></a>

Quick Start

<a name="quick-installation"></a>

Usage

You can directly copy a command from the typical use cases below to run Ailice.

ailice   #Use models configured individually for different agents under the agentModelConfig field in config.json.
ailice_web --speechOn=1 --ttsDevice=cuda --sttDevice=cuda
ailice --modelID=anthropic:claude-sonnet-4-20250514 --contextWindowRatio=0.2
ailice --modelID=openrouter:z-ai/glm-4.5 --chatHistoryPath=./chat_history --contextWindowRatio=0.2
ailice --modelID=mistral:mistral-large-latest --prompt="researcher"
ailice --modelID=deepseek:deepseek-chat
ailice --modelID=hf:Open-Orca/Mistral-7B-OpenOrca --quantization=8bit --contextWindowRatio=0.6
ailice --modelID=groq:llama3-70b-8192
ailice --modelID=openrouter:google/gemini-2.5-pro
ailice --modelID=lm-studio:qwen2-72b --contextWindowRatio=0.5

It should be noted that the last use case requires you to configure the LLM inference service first, please refer to How to Add LLM Support. Using inference frameworks such as LM Studio can use limited hardware resources to support larger models, provide faster inference speed and faster Ailice startup speed, making it more suitable for ordinary users.

When you run it for the first time, you will be asked to enter the api-key. You can also modify the api-key by editing the config.json file. Please note that the first time When using an open source LLM, it will take a long time to download the model weights, please make sure you have enough time and disk space.

When you turn on the speechOn switch for the first time, you may need to wait for a long time at startup. This is because the weights of the speech recognition and TTS models are being downloaded in the background.

As shown in the examples, you can use the Agent through ailice, it provides a web dialogue interface. You can view the default value of each parameter by using

ailice --help

The default values for all command line arguments can be customized by modifying the corresponding parameters in config.json.

- --modelID There are two modes for model configuration. In the first mode, the model is uniformly specified by modelID. In the second mode, different types of agents will run on different models. When this parameter is an empty string (unspecified), the second mode will be used automatically, i.e., the models configured individually for different agents under the agentModelConfig field in config.json will be used, for details please refer to Using Different Models in Different Agents. The currently supported models can be seen in config.json. - --quantization is the quantization option, you can choose 4bit or 8bit. The default is not quantized. - --maxMemory is the memory video memory capacity constraint, the default is not set, the format when set is like "{0:"23GiB", 1:"24GiB", "cpu": "64GiB"}". - --prompt specifies the prompt to be executed, which is the type of agent. The default is 'main', this agent will decide to call the appropriate agent type according to your needs. You can also specify a special type of agent and interact with it directly. - --temperature sets the temperature parameter of LLM reasoning, the default is zero. - --flashAttention2 is the switch to enable flash attention 2 to speed up inference. It may have a certain impact on output quality. - --contextWindowRatio is a user-specified proportion coefficient, which determines the proportion of the upper limit of the prompt length constructed during inference to the LLM context window in some cases. The default value is 0.6. - --speechOn is the switch to enable voice conversation. - --ttsDevice specifies the computing device used by the text-to-speech model. The default is "cpu", you can set it to "cuda" if there is enough video memory. - --sttDevice specifies the computing device used by the speech-to-text model. The default is "cpu", you can set it to "cuda" if there is enough video memory. - --resetApiKey whether to reset the model's API key after startup. - --chatHistoryPath is used to specify the directory where chat history data is stored. - --certificate Certificate settings for the web interface. The simplest option is an empty string, which will use the HTTP protocol for the UI web page. Setting it to 'adhoc' will use a self-generated certificate, providing encryption for the data flow between the UI and server, but it requires dismissing browser security warnings. The most secure method is to apply for a certificate and set this parameter to '{"cert": "your_cert.pem", "key": "your_key.pem")'. - --expose Whether to provide public access. - --share create a publicly shareable link for Ailice. (For security reasons, we have temporarily removed this feature. It will be re-enabled once more security measures are implemented in the UI. Please ensure that the services provided by app.py are not exposed to any untrusted networks)

<a name="configuring-extension-modules-and-mcp-servers"></a>

Guide to Choosing an LLM

Updated on Aug 15, 2025.

Currently, Ailice can handle more complex tasks using the locally run 72B open-source model (qwen-2.5-72b-instruct running on 4090x2). Considering the low cost of open-source models, we highly recommend users to start using them. Moreover, localizing LLM operations ensures absolute privacy protection, a rare quality in AI applications in our time. Click here to learn how to run LLM locally. For users whose GPU conditions are insufficient to run large models, you can use the online inference service (such as openrouter, this will be mentioned next) to access these open-source models (though this sacrifices privacy). You can make agents excel by leveraging different models according to their strengths and weaknesses. For details, please refer to Using Different Models in Different Agents.

claude-3-5-sonnet/claude-3-7-sonnet/claude-sonnet-4/gemini-2.5-pro provides the best performance.

z-ai/glm-4.5 is very close to top-tier performance.

qwen-2.5-72b-instruct provides the best performance at the 70B level

gpt-4-turbo/gpt-3.5-turbo is surprisingly lazy, and we have never been able to find a stable prompt expression. gpt-4o used to have top-tier performance, but currently (similar to the previous turbo models) they exhibit laziness issues in function calls. We no longer recommend using them.

For users whose hardware capabilities are insufficient to run open-source models locally and who are unable to obtain API keys for commercial models, they can try the following options:

• openrouter/apipie These services can route your inference requests to various open-source or commercial models without the need to deploy open-source models locally or apply for API keys for various commercial models. They're fantastic choices. Ailice automatically supports all models in OpenRouter/Apipie. Thanks @babybirdprd for recommending OpenRouter to me.

<a name="the-most-outstanding-open-source-model"></a>

Configuring Extension Modules and MCP Servers

The configuration file of Ailice is named config.json, and its location will be output to the command line when Ailice is started. In this section, we will introduce how to extend Ailice's external interaction capabilities through configuration file.

In Ailice, we use the term "module" to specifically refer to components that provide functions for interacting with the external world. Each module runs as an independent process; they can run in different software or hardware environments from the core process, making Ailice capable of being distributed. We provide a series of basic module configurations in the configuration file required for Ailice's operation (such as vector database, search, browser, code execution, etc.). You can also add configurations for new modules. Module configuration is very simple, consisting of only two items:

  "services": {
    ...
    "scripter": {"cmd": "python3 -m ailice.modules.AScripter --addr=tcp://127.0.0.1:59000",
	               "addr": "tcp://127.0.0.1:59000"},
    ...
  }

Among these, under "cmd" is a command line used to start the module's process. When Ailice starts, it automatically runs these commands to launch the modules. Users can specify any command, providing significant flexibility. You can start a module's process locally or utilize Docker to start a process in a virtual environment, or even start a remote process. Some modules have multiple implementations (such as Google/Storage), and you can configure here to switch to another implementation.

"addr" refers to the address and port number of the module process. Users might be confused by the fact that many modules in the default configuration have both "cmd" and "addr" containing addresses and port numbers, causing redundancy. This is because "cmd" can, in principle, contain any command (which may include addresses and port numbers, or none at all). Therefore, a separate "addr" item is necessary to inform Ailice how to access the module process.

Ailice can use tools from various MCP servers, simply by wrapping the MCP server with the ailice_mcp_wrapper command, which allows it to be used as a standard extension module for Ailice.

For an MCP server started locally in stdio mode, assuming the startup command is mcp_echo hello, we use the following configuration to launch it as a standard Ailice service.

  "services": {
    ...
    "mcp_echo": {"cmd": "ailice_mcp_wrapper --addr tcp://127.0.0.1:59200 stdio mcp_echo hello",
                 "addr": "tcp://127.0.0.1:59200"},
    ...
  }

For an MCP server in SSE mode, assuming its service address is: http://example:8000/sse, we use the following configuration to connect to it.

  "services": {
    ...
    "mcp_echo": {"cmd": "ailice_mcp_wrapper --addr tcp://127.0.0.1:59200 sse --server_url http://example:8000/sse",
                 "addr": "tcp://127.0.0.1:59200"},
    ...
  }

For extension modules that are set up in the configuration file, Ailice automatically loads them after startup and selects the appropriate tools for the current task. In addition to using config.json, you can also provide Ailice with the address and port of the extension module process during runtime, and it can dynamically load and use the module.

<a name="useful-tips"></a>

Selection and Configuration of LLM

<a name="guide-to-choosing-an-llm"></a>