Configuration: How to run examples with other LLM backend

[thorsmann]

Hello, I am playing around with the healthcare demo (https://www.parlant.io/docs/quickstart/examples/).

How can I override the base URL of the OpenAI endpoint wrapper? We use a variety of models which are all wrapped behind the OpenAI chat protocol. I would need some way to override the base URL of the OpenAI server object.

I try to set OPENAI_PROVIDER_BASE_URL accordingly and used a valid and working api key for our own endpoint but I get this error message: {'error': {'message': 'Incorrect API key provided:

It seems like there is some hard-wired authentication against the actual OpenAI service (which obviously does not know our API key). Did I miss something obvious?

[kichanyurd]

Hey @thorsmann,

While we have plans to add what should be properly called an OpenAICompatibleService, we need to specify it and understand how it would work. With Parlant, since it's not only a matter of "connecting" to a service but, much more than that, how to reliably use it at scale with high SLAs, this means that simply changing the base URL or choosing a model won't be enough for more than mere prototyping.

And production reliability is what we're all about.

With that said, Parlant places very few hard limitations on you as a developer — as a strict design goal.

In fact, there's a documentation page I want to refer you to — on custom LLMs — which takes you step by step into implementing your own NLPService. The easiest thing is indeed to just copy the baseline openai_service.py implementation and make the necessary changes there — and then import it manually into your project (as described in our docs).

If you run into any issues or have any questions, please don't hesitate at all to bring them up either here or on Discord!

[hjc3613]

how to compatible with dashscope ie.. qwen official api

[baijiu-in-my-cup]

custom_service.py
hi,this is how I got started quickly.

In addition,I want to express some thoughts!I am a tech newbie.This project is very powerful, and it also makes me feel that it is very complex.o(╥﹏╥)o

[kichanyurd]

custom_service.py
hi,this is how I got started quickly.

In addition,I want to express some thoughts!I am a tech newbie.This project is very powerful, and it also makes me feel that it is very complex.o(╥﹏╥)o

Yes, and it will make you more powerful too :D

But yeah the team is made up of some pretty experienced software developers and we've definitely put our whole experience into Parlant's design.