Last week, I realized something I’ve always cautioned against: using overly rigid prompts for our LLM-driven customer support bot. Essentially, we were taking a high-performance vehicle and forcing it onto a single-track rail, unable to maneuver freely where it excelled most.

The “Aha” Moment

During a recent review, we encountered a user seeking to reschedule an appointment, complicated by an international trip to a different time zone. Our meticulously defined prompt structure gave a formally correct but practically useless answer about local time slots—completely ignoring their unique travel scenario.

Below is our original setup:

# Before: Overly Restrictive
def handle_scheduling(request)
  response = llm.complete(
    prompt: "TIME_SLOT_TEMPLATE",
    parameters: {
      date: request.date,
      time: request.time,
      action: "RESCHEDULE"
    }
  )
end

Now, here’s the updated version:

# After: Letting AI Shine
def handle_scheduling(request)
  response = llm.complete(
    prompt: "You're a helpful scheduling assistant. The customer needs assistance with: #{request}. 
            Consider their situation, time zones, and preferences, while keeping a natural dialog."
  )
end

The change was dramatic. Instead of rigidly following a template, our AI started asking the right questions, understanding user circumstances, and even coming up with alternative suggestions we hadn’t considered before.

Why Does This Happen?

It’s understandable. We often want to control every detail to ensure reliability and predictability. Sadly, this approach ends up:

1. Neglecting the most powerful feature of LLMs—their capacity to interpret context and adapt.
2. Piling extra work on our shoulders.
3. Diminishing the user’s overall experience.

Breaking Free: What Works Better

After months of trial and error, here’s what made the difference:

1. Trust the Process
   Instead of micromanaging each interaction, clarify goals and boundaries and let the AI navigate how to reach them.

2. Embrace Variation
   Users differ in preferences, tone, and needs. Modern LLMs excel at customizing responses if allowed freedom.

3. Measure Success, Not Layout
   Concentrate on whether solutions are effective, not whether they follow a rigid script.

A Real-World Example

Here’s an example from our appointment booking process:

# Old Method: Step-by-Step Script
def book_appointment
  steps = [
    "ASK_DATE",
    "VALIDATE_DATE",
    "ASK_TIME",
    "VALIDATE_TIME",
    "CONFIRM_BOOKING"
  ]
  
  steps.each do |step|
    execute_step(step)
  end
end

# New Method: Natural Interaction
def book_appointment
  context = {
    available_slots: get_available_slots(),
    user_timezone: detect_user_timezone(),
    scheduling_preferences: get_user_preferences()
  }
  
  llm.interact(
    system_prompt: "Help schedule an appointment in the best possible way. 
                   Be mindful of the user’s time zone and preferences.",
    context: context
  )
end

Taking It Forward

Whenever you’re setting up an LLM-based solution, ask yourself: “Am I empowering the AI or just imposing a strict checklist?”

Our ultimate goal is to solve real-world problems and support users effectively. Sometimes, giving the AI more freedom is exactly what’s needed to accomplish that.