Arc Assistant Customer Questions

[nstevens]

1. Is Arc Assistant still only compatible with OpenAI services? If so, is there a reason for this? (We wouldn’t be able to connect to any external AI services and would have to use our in-house one, so trying to understand the compatibility requirements)

2. How does Arc Assistant understand the TM1 specific syntax of rules & TIs etc? Is that built into the Arc application, or would it rely on the AI service to do that? Would we need to embed anything into the internal AI service to make it work correctly?

[harvey]

Hi @nstevens,

Arc never only supported OpenAI models. Arc has always supported LLMs that use the OpenAI protocol.

From the web:

Beyond OpenAI’s own models (such as GPT-4.1 and GPT-4o), a wide range of proprietary and open-source Large Language Models utilize the OpenAI API protocol, either natively or through compatible inference servers.

Major Proprietary Models

• Anthropic Claude: Models like Claude 3.5 Sonnet and Opus 4.6 are frequently accessed via OpenAI-compatible endpoints through gateways like Helicone, Ofox, or AWS Bedrock.

• Google Gemini: Models such as Gemini 2.5 Flash and Pro can be accessed via OpenAI-compatible APIs through Google Vertex AI or wrappers like Any-LLM.

• Mistral Models: The Mistral Large 3 and open-weight variants (e.g., Mixtral 8x7B) are served via OpenAI-compatible APIs by providers like Mistral AI, Groq, and Together AI.

• Meta LLaMA: The LLaMA 3 series is available through OpenAI-compatible endpoints on platforms like AWS Bedrock, Azure OpenAI, and SiliconFlow.

• DeepSeek: The DeepSeek R1 model is accessible via OpenAI-compatible APIs, often through providers like DeepSeek’s own API or third-party aggregators.

Open-Source and Local Models

• Local Inference Servers: Tools like Ollama, vLLM, LM Studio, and Jan serve local models (including LLaMA, Mistral, and Qwen) using a local server that mimics the OpenAI API structure, allowing applications to interact with them as if they were remote services.

• Managed Open-Source Providers: Services like SiliconFlow, Hugging Face Inference Endpoints, and Fireworks AI host various open-weight models (e.g., Qwen, Yi) behind OpenAI-compatible API interfaces.

Unified Gateways Platforms such as LiteLLM, Helicone, and Ofox act as intermediaries, providing a single OpenAI-compatible API that routes requests to dozens of different underlying providers (including Anthropic, Google, and various open-source models), abstracting away the differences in their native API formats.

Info on how to config here: Using Arc Assistant with WatsonX, Google Gemini, Groq or any OpenAI compatible service

---

Depending what model you use, it might be trained quite well on TM1. Claude does a pretty good job with some direction, and Arc is always improving its internal prompting (what information Arc gives the LLM in addition to your typed question) to ensure LLMs get more and more useful with each release.

However, we also support setting up an embeddings model to augment your LLM’s TM1 knowledge. Embeddings organize text-based info (in this case, TM1 documentation) in a vector format that’s easy for LLMs to consume.

There’s a post about setting it up below:

Arc v5 AI Assistant Embedding Model

Hi All, I know the Arc AI assistant is able to be set up with an embeddings server as of version 5. Do we have any guidance on doing so documented yet? Any basic steps outlined that we can take to set up the embeddings server/service correctly? Thank you, Nikita

[Vincent]

@harvey

Arc has always supported LLMs that use the OpenAI protocol.

Thanks @harvey for the detailed answer.

I updated the documentation to make it clearer that Arc Assitant can connect to most LLM providers as along as they support OpenAI protocol:

Enable the Arc Assistant - Arc User Manual - Confluence

[Vincent]

@nstevens ,

Harvey replied to question 2 in the last part of this answer:

@harvey

Depending what model you use, it might be trained quite well on TM1. Claude does a pretty good job with some direction, and Arc is always improving its internal prompting (what information Arc gives the LLM in addition to your typed question) to ensure LLMs get more and more useful with each release.

However, we also support setting up an embeddings model to augment your LLM’s TM1 knowledge. Embeddings organize text-based info (in this case, TM1 documentation) in a vector format that’s easy for LLMs to consume.

There’s a post about seeing it up below:

[Vincent]

Sounds good to me. What is important is that their internal AI service is OpenAI compatible service.