The problem TruthScope solves

TruthScope

TruthScope is an evidence-first claim-verification prototype built for Malaysian public-interest and political information.

Problem statement

Public claims spread through speeches, news, social media posts, forwarded messages, and screenshots faster than most people can investigate them. Checking one claim often requires users to:

• find original sources among repeated or copied reporting;
• distinguish direct records from commentary;
• check dates, quotation context, units, and omitted qualifiers;
• compare sources that support and contradict the claim;
• judge whether an AI answer is evidence-based or merely confident; and
• understand why models disagree.

Political claims add another risk: source identity, party affiliation, office, race, religion, or popularity can become shortcuts for credibility. A system that produces one unexplained “true/false” label may hide uncertainty and reproduce those shortcuts.

Proposed solution

TruthScope helps users investigate a claim instead of asking them to trust one model. It:

1. extracts atomic, verifiable claims;
2. gathers public source pages and preserves provenance;
3. checks date, quotation, and statistical context;
4. asks two distinct Gonka-hosted verifier models to assess the same evidence;
5. uses a third distinct model to compare agreement and disagreement;
6. audits judgment language for political-bias indicators;
7. calculates deterministic evidence and confidence scores; and
8. shows sources, limitations, failures, and Gonka request metadata to the user.

Challenges we ran into

Challenges We Ran Into

• Unstable AI provider responses: Gonka models sometimes timed out, returned HTTP 400 errors, or produced invalid structured output. We added task-specific timeouts, bounded retries, output validation, graceful degradation, and different models for independent verification.

• Slow verification pipeline: Multiple sequential AI calls made results take too long. We reduced unnecessary calls, kept two independent verifiers plus one consensus judge, and added a resumable background-job API with frontend polling.

• Progress disappeared after refresh: Verification originally depended on one browser request. We stored the active job ID locally and resumed polling after refresh while backend processing continued.

• Missing completed history records: Gonka could route two verifier requests to the same served model, conflicting with a database uniqueness rule and rolling back persistence. We changed inference uniqueness to use Gonka request IDs and improved persistence error logging.

• History showed only some records: Frontend loaded paginated history but rendered only a small batch. We now fetch all user-owned pages and render every loaded record with search and verdict filters.

• Evidence retrieval configuration: Text claims required live public sources, but search was initially unavailable. We integrated Brave Search, added configuration health reporting, and retained safe fallback behavior when search fails.

• OAuth redirect problems: Google and GitHub authentication could return users to unreachable localhost URLs. We aligned provider callback URLs, Supabase Site/Redirect URLs, and frontend OAuth redirect configuration.

• Frontend and backend authorization: Requests returned HTTP 401 when browser sessions were not forwarded correctly. We attached Supabase Bearer tokens, refreshed expired sessions once, and kept backend ownership checks and Supabase RLS as security boundaries.

• Multilingual results: Translating only frontend labels left AI reports in English. We added English, Bahasa Melayu, and Simplified Chinese throughout the UI and passed the selected output language through the verification pipeline.

• Container and AWS deployment: ECR tags, IAM permissions, runtime environment files, and stale local image tags caused deployment confusion. We adopted exact immutable release tags, separate frontend/backend images, health checks, and documented rebuild, push, pull, and recreate steps.

• Public HTTPS without a domain: Direct EC2 ports were unsuitable for OAuth and public testing. We used separate Cloudflare Quick Tunnels for frontend and backend, then restricted EC2 inbound rules while documenting the temporary-URL limitation.

• Trust and transparency: AI results must not be presented as guaranteed truth. We exposed evidence sources, model disagreements, confidence, limitations, Gonka request IDs, and clearly labelled Evidence Support as a support indicator rather than a probability of truth.