Ash Pembroke, Portfolio CTO of Caylent, discusses the critical balance of data accuracy in the era of Gen AI for the benefit of boosting innovation.

Topics Include:

• Ash Pembroke, Portfolio CTO of Caylent, self-identifies as a "recovering data scientist."
• Caylent is an AWS native services company.
• Data quality remains an issue despite Gen AI.
• Contrasts legalism versus mysticism in data quality.
• Legalism: accurate data when applications need it.
• Mysticism: insights that help decision-making.
• Traditional data foundations approach is being challenged weekly.
• Gen AI developments force rethinking of solution architectures.
• Teams share solutions through giant Slack threads.
• Example: Vector databases questioned after model context protocol.
• Still do traditional data assessments, but stay flexible.
• Integration and data processing constantly get abstracted.
• Data strategy equals architecture strategy equals business strategy.
• Traditional approach: standardize data across engineering teams.
• New approach: allow business users to innovate.
• Bring valuable techniques back to the organization.
• Case study: North Sea wind turbine alerts.
• Initially seen as data quality issue, revealed new predictive failure signal.
• Gen AI enables local experimentation by business users.
• Blurring lines between enterprise enablement and software building.
• BrainBox AI case study: energy optimization across buildings.
• Architecture decisions impact ability to scale products.
• Work with business edges rather than looking for patterns.
• Gen AI can process information from these working groups.
• Think about data as a product, not asset.
• Redimensionalize dependencies across your organization.
• Now's a good time to attack data quality.
• New tools help visualize complexity across organizations.

Participants:

· Ash Pembroke – Portfolio CTO, Caylent

See how Amazon Web Services gives you the freedom to migrate, innovate, and scale your software company at https://aws.amazon.com/isv/

New Relic's Head of AI and ML Innovation, Camden Swita discusses their four-cornered AI strategy and envisions a future of "agentic orchestration" with specialized agents.

Topics Include:

• Introduction of Camden Swita, Head of AI at New Relic.
• New Relic invented the observability space for monitoring applications.
• Started with Java workloads monitoring and APM.
• Evolved into full-stack observability with infrastructure and browser monitoring.
• Uses advanced query language (NRQL) with time series database.
• AI strategy focuses on AI ops for automation.
• First cornerstone: Intelligent detection capabilities with machine learning.
• Second cornerstone: Incident response with generative AI assistance.
• Third cornerstone: Problem management with root cause analysis.
• Fourth cornerstone: Knowledge management to improve future detection.
• Initially overwhelmed by "ocean of possibilities" with LLMs.
• Needed narrow scope and guardrails for measurable progress.
• Natural language to NRQL translation proved immensely complex.
• Selecting from thousands of possible events caused accuracy issues.
• Shifted from "one tool" approach to many specialized tools.
• Created routing layer to select right tool for each job.
• Evaluation of NRQL is challenging even when syntactically correct.
• Implemented multi-stage validation with user confirmation step.
• AWS partnership involves fine-tuning models for NRQL translation.
• Using Bedrock to select appropriate models for different tasks.
• Initially advised prototyping on biggest, best available models.
• Now recommends considering specialized, targeted models from start.
• Agent development platforms have improved significantly since beginning.
• Future focus: "Agentic orchestration" with specialized agents.
• Envisions agents communicating through APIs without human prompts.
• Integration with AWS tools like Amazon Q.
• Industry possibly plateauing in large language model improvements.
• Increasing focus on inference-time compute in newer models.
• Context and quality prompts remain crucial despite model advances.
• Potential pros and cons to inference-time compute approach.

Participants:

• Camden Swita – Head of AI & ML Innovation, Product Management, New Relic

See how Amazon Web Services gives you the freedom to migrate, innovate, and scale your software company at https://aws.amazon.com/isv/

Jon Yoo, CEO of Suger, shares how his company automates the complex & challenging workflows of selling software through cloud marketplaces like AWS.

Topics Include:

• Jon Yoo is co-founder/CEO of Suger.
• Suger automates B2B marketplace workflows.
• Handles listing, contracts, offers, billing for marketplaces like AWS.
• Co-founder previously led Confluent's marketplace enablement product.
• Confluent had 40-50% revenue through cloud marketplaces.
• Required 10-20 engineers working solely on marketplace integration.
• Engineers prefer core product work over marketplace integration.
• Product/engineering leaders struggle with marketplace deployment requirements.
• Marketplace customers adopt without marketing, creating unexpected management needs.
• Version control is challenging for marketplace-deployed products.
• License management through marketplace creates engineering challenges.
• Suger helps sell, resell, co-sell through AWS Marketplace.
• Marketplace integration isn't one-time; requires ongoing maintenance.
• Business users constantly request marketplace automation features.
• Suger works with Snowflake, Intel, and AI startups.
• Data security concerns drive self-hosted AI deployments.
• AI products increasingly deploy via AMI/container solutions.
• AI products use usage-based pricing, not seat-based.
• Usage-based pricing creates complex billing challenges.
• AI products are tested at unprecedented rates.
• Two deployment options: vendor cloud or customer cloud.
• SaaS requires reporting usage to marketplace APIs.
• Customer-hosted deployment simplifies some billing aspects.
• Marketplaces need integration with ERP systems.
• Version control particularly challenging for AI products.
• Companies need automated updates for marketplace-deployed products.
• License management includes scaling up/down and expiration handling.
• Suger aims to integrate with GitHub for automatic updates.

Participants:

· Jon Yoo – CEO and Co-founder, Suger

See how Amazon Web Services gives you the freedom to migrate, innovate, and scale your software company at https://aws.amazon.com/isv/