It is 11 p.m. A prospect opens a 360° tour long after the office has closed. They ask about a room, the AI answers inside the tour and takes them to the relevant scene. They use voice input for a follow-up question. Once their interest becomes specific, they share their contact details in the conversation. A late-night visit turns into an enquiry, handled by an AI concierge for virtual tours without a team member watching the chat.
Panomity enables that journey by adding a self-hosted AI concierge to a virtual tour. It turns a sequence of panoramas into a two-way experience. Visitors can ask, explore and take the next step in one place. The underlying AI runs on Panomity’s own infrastructure in Germany.
The foundation is CMS4VR at vr.panomity.com, a Krpano-based 360° tour platform. Its AI stack is 100% self-hosted on Panomity’s own servers and GPU in Germany. Chat uses no US cloud API. Voice input, Vision processing and GPUQ also form part of the company’s own infrastructure.
What is an AI concierge for virtual tours?
An AI concierge for virtual tours is a multilingual assistant embedded in a digital walkthrough. It answers questions from the scene catalogue and AI-generated descriptions, guides visitors to relevant views, launches a guided tour and can collect contact details. Panomity’s concierge performs these tasks within a fully self-hosted AI stack.
The spatial context separates it from a detached website chatbot. The concierge knows the tour’s scene catalogue and can use control markers to connect an answer with movement through the venue. Panomity provides more about the format in Virtual Tour Magic: Experience 360 Degrees of Success.
That model can support several kinds of organisation:
- Estate agents guide prospects through houses and apartments.
- Hotels answer questions about the areas shown in a tour.
- Museums accompany visitors through digital exhibitions.
- Retail stores make visible spaces part of a conversation.
- Agencies create guided tour experiences for client projects.
How does the AI concierge work?
The concierge is an Ollama-based LLM chat inside the Krpano-based CMS4VR platform. Its server builds a system prompt from the scene catalogue and AI scene context. Replies stream as NDJSON. Panomity’s GPUQ coordinates AI jobs and reports queue position, status heartbeats and estimated waiting time to the front end.
How does it produce a relevant answer?
The server supplies the available scene information to the local language model as context. The model running through Ollama turns that context into a multilingual response. Because the system prompt is assembled server-side, each answer can draw on the catalogue and automatically generated descriptions for that particular tour.
Control markers connect language to navigation. [[GOTO:pID]] tells the tour to switch to a specific scene, while [[TOUR:START]] launches a guided tour. When a visitor provides contact details, [[LEAD:{json}]] passes them to the lead function. The markers trigger scene changes, the tour start or lead handover directly from chat.
What happens when several AI jobs are waiting?
GPUQ fairly shares a single GPU on Panomity’s own infrastructure across chat, Vision, STT, Depth and SHARP jobs. The queue sends status heartbeats, the current position and an ETA to the front ends. Visitors can therefore see that work is in progress while several AI services use the same self-hosted infrastructure.
Chat responses arrive as an NDJSON stream, allowing the interface to present output as it is generated while also displaying queue information. The architecture brings several AI workloads together on one GPU server. It does not need to send the processing to a US cloud AI provider.
How does voice input work without cloud AI?
Voice input uses Panomity’s own faster-whisper microservice. The “small” model runs on the CPU with int8 quantisation, FastAPI and a VAD filter. The service is available only locally on port 8032. Recorded audio never leaves the server; the recognised question passes straight into the concierge conversation.
A visitor can ask a question aloud while viewing a scene. The local service converts the recording to text without sending the audio away from the server. The concierge then processes the recognised question with the same scene context used for a typed message.
The VAD filter identifies speech segments in the recording. The entire speech-to-text component remains within the self-hosted stack. That matters when you want to offer a spoken interface without sending the visitor’s audio to an external cloud AI service.
How does the AI know every scene?
A Vision function automatically describes every 360° panorama. Panomity stores each description by language and adds it to the concierge context. The platform also generates conversation starters for individual scenes and creates Points of Interest for all visible scenes in batches through the GPU queue. Together, these elements give the dialogue a specific spatial reference.
Each component has a distinct job:
- Scene descriptions give the concierge a written context for each panorama.
- Conversation starters offer visitors a relevant way into the dialogue in each scene.
- Automatic POIs identify Points of Interest for all visible scenes.
The Vision function processes the tour’s 360° panoramas. Panomity provides more information about 360-degree images. For the concierge, the automatically generated descriptions are stored by language. The server combines them with the scene catalogue to create AI scene context.
How does a question become a lead?
The concierge supports the visitor from an initial question about the venue through to contact in the same conversation. When interest becomes concrete, it can collect contact details and pass them to the lead function with the designated marker. Contact details are therefore captured directly in the dialogue inside the virtual tour.
The sequence is straightforward. A visitor explores a scene, asks a question and receives a context-aware reply. The concierge can then change the scene or begin the guided tour. If the visit develops into an enquiry, it collects the contact details in the dialogue. This is how the AI concierge can support visitors around the clock: it answers, guides and can capture a lead, including at 11 p.m.
Why is self-hosting a GDPR advantage?
Self-hosting keeps the described AI stack on Panomity’s own infrastructure in Germany and preserves data sovereignty. Chat, voice input and AI processing require no US cloud AI. That makes the approach GDPR-friendly. Voice input provides a particularly clear example: its audio is processed locally and never leaves the server.
The stack includes Ollama chat, the local faster-whisper service, Vision jobs and GPUQ. Rather than calling separate external AI APIs for those tasks, the services run on Panomity’s own servers and GPU. This provides a direct technical basis for data sovereignty.
“GDPR-friendly” is the precise description here. The basis is a fully self-hosted stack, Panomity’s own servers and its own GPU in Germany. The AI features described transfer no data to US cloud AI, while Panomity retains full data sovereignty.
AI concierge FAQ
Is the AI concierge fully self-hosted?
Yes. Panomity’s AI stack runs on its own servers and GPU in Germany. Chat uses Ollama, while voice input uses a local faster-whisper service. These functions send no data to US cloud AI, so Panomity retains technical data sovereignty over the stack.
Can the concierge control the virtual tour?
Yes. Internal control markers let the concierge switch to a specified scene or launch a guided tour. Its response is therefore not limited to text: it can take visitors directly to a relevant view and continue the walkthrough as part of the ongoing conversation.
How does voice input work without the cloud?
A local faster-whisper microservice converts speech into text. Its “small” model runs on the CPU with int8 quantisation and a VAD filter. The FastAPI service is reachable only through localhost. Audio never leaves the server; the recognised text is then processed as a question.
How does the concierge learn about each scene?
The Vision function describes every 360° panorama automatically and stores the results by language. The server combines those descriptions with the scene catalogue to form the AI context. The platform also generates conversation starters per scene and Points of Interest for all visible scenes.
How does the AI concierge capture a lead?
A visitor can provide contact details directly in the conversation. The concierge passes those details to the lead function through an internal lead marker. An answered question can therefore become a specific enquiry within the virtual tour without making the visitor leave the active walkthrough.
Conclusion: A tour that continues the conversation
The self-hosted AI concierge connects 360° scenes, local speech processing, navigation and lead capture. It responds in multiple languages and can support visitors around the clock by answering questions, guiding visitors and offering a contact path, without relying on US cloud AI. Explore how Panomity approaches virtual reality and VR tours if you want your walkthrough to become an active conversation.
0 Comments