← Lab

Can Jev build structured objects without generating text?

Result

Jev produced structured objects 4× faster than GPT-6 Luna: 418 ms versus 1687 ms median time. It produced 100% schema-valid objects, with 95.8% exact matches. Luna reached 100% exact matches and cost less.

A valid object can still be wrong

  • Jev, typed choices95.8%
  • Luna, strict output100%
  • Mixed, Jev + Luna95.8%
Each square is one of 24 requests. Filled squares match the whole object; outlined squares pass the schema but get something wrong. Jev and Mixed both emailed twice instead of waiting, then emailing.

Time to produce an object

  • Jev, typed choices418 ms

    $0.444 per 1000 objects

  • Luna, strict output1687 ms

    $0.171 per 1000 objects

  • Mixed, Jev + Luna1963 ms

    $0.299 per 1000 objects

Median time including model calls, assembly and validation. Mixed saves 33% versus all-Jev but costs 1.75× Luna and does not fix the wrong branch.

Methodology

Give each method the same 24 synthetic requests, spanning six schema levels from flat fields to nested workflows. Jev selects typed values and references from supplied memory; Luna generates the whole object; Mixed uses Jev for closed choices and Luna for open fields. Allow one attempt with no repair. Validate each object against its schema, then check every value and branch against the answer key. Mixed ran later, so its timing and caching are not a controlled paired comparison.

← All findings