Hybrid agentic inference is coming soon to Perplexity Computer: What is it

As AI models become more capable, companies are looking for ways to balance performance, privacy, and the rising cost of compute. While cloud-based models offer greater processing power, they require data to be sent to remote servers. On-device AI can keep information local, but is often constrained by hardware limitations. Determining which workloads should run locally and which should be handled in the cloud has emerged as what the industry increasingly describes as an “orchestration problem.”

What is an orchestration problem?

An orchestration problem is the challenge of deciding which AI model should do which part of a task, where it should run, and when. In Perplexity’s case, imagine you’re asking an AI to analyse your bank statement and create a financial summary. 

Some parts of the task involve sensitive personal data that should ideally stay on your laptop, while other parts may require the reasoning power of a larger cloud-based AI model. The orchestration problem is figuring out how to split the work between the local…