[music]

Well, thank thank you all. Um, let me

get set up here. So, not just the the

founder of Arise, but but I tend to

build an incredible amount of stuff. Um,

let's see if we get this going here.

Oh, sorry. One more second. Um, so not

just a founder here, but but also a

builder and I do my best to um uh to to

to build agents assistance. Um, we have

an agent in product.

We have an agent in product called Alex

and uh and a lot of I think a lot of my

experience has come from actually um

trying to make the stuff work and work

well. Our first version of our our own

agent frankly sucked. Uh it was many

years ago uh probably two years ago when

the first in the space to do it. Um and

a lot of what we have built uh has come

out of that our own experience in

building building this agent and and

signal is kind of our our next

generation of this which is trying to

automate a bunch of things which we do

every day uh and build it into products

that that people can use. Um so I'm

going to try to I'm going to go through

this this materials here. I'll try to go

fast and try to show you a lot of

product too. I'm a product person. Um,

so

if you've built a startup before, um,

you you've experienced this, your

platform's down, it's it's late at night

and and you and you want to go fix it.

Um, and and really the the it takes a

lot of energy to go do that. And we're

going to talk about like the automation

we've built a little bit and and what

what the future looks like and and I

truly believe um that the future of the

the observability space is is actually

changing massively right now. Why is

that? Well, observability used to be for

humans. Used to be a UI you click, a

graph you click, something you look at.

Um and and today it's I would argue it's

a lot of 2.0 which is like this

combination of coding agent. Those of

you who built skills skills for um

Pyroscope, Google Cloud or or whatnot,

that these these skills help you with

your your human debugging these systems.

Um and and really telemetry is like this

smoke uh thrown off of your system that

can allow these agents to go make fixes.

It tells you what path in the code it

took. Without that, you're guessing and

there's a million paths it could have

taken. the the data thrown off by your

system allows um allows you to to go go

use agents to go debug your software.

Evals add another layer to this. Um but

really what we're at here is is how do I

build systems that autonomously fix

themselves? Really that that is what

we're after. Both both AI agents, I put

AI into my my my system. How do I have

this thing just improve itself? and and

today we're kind of in the 2.0 which is

a human making fixes and reviewing

things. Um but there's a future we're

all driving towards and throwing off

traces, throwing off logs, throwing off

way more than you normally would and

having agents run at this for a

continuous loop is where we're going.

You can build at agent speed, but today

you can't improve your systems really at

this agent speed. So those of us feel

this this kind of governor happening

within our our our our products. um and

and the bottleneck is actually not the

fix anymore. So those of us who've used

these systems and and use used um coding

agents with with skills, the the the

bottlenecks, a lot of the the confidence

in in do I have it right? You know, a

lot of this is is about is this fix the

right one to push? Um and and so these

are kind of the challenges here. And

then how do you how do you build this

loop in a way that just moves faster? Um

and and a little bit of the way we we've

kind of come to do it and we do it in

our system is we've kind of inverted

this this loop which is like a human you

know looks at things and an agent uh

fixes it to a person now can wake up

with with an idea of the issues based

upon the errors occurred in their

system. So so the agent is actually you

know maybe it's not a a fix itself but

it's putting up an issue. It's looking

at the data before a human even looks at

it. Um and and what you move from there

is is kind of humans grabbing tickets to

to having some amount of evidence um

some deep evidence relative to whatever

you're looking at already sitting in

front of you by the time you actually

even look at it.

And and human review is kind of one

thing but but a lot of times maybe

you're driving this little investigation

a bit from where it started. So that's

the reality of where we are today is

there's still maybe it's not human

reviewing but human driving the the step

two and three. Um but but this is kind

of what what we view the loop as. And

really what it is is there's you know

there's an event that occurs that you're

kind of kicking things off on or you're

looking at periodically. Um and then

there's some context around that which

is really driven by skills. Um, I guess

a question for all of you. Who's created

skills in this room? Who's created a

skill that that that interfaces to an

observability platform?

Okay, handful. Okay, cool. Awesome. Um,

so the magic of of of skills that that

that connect to observability platforms

um is it can gather the context. The

agent can decide what it needs, what it

needs to look at um to to start to

troubleshoot what you have there. Um,

and then there's idea of triggers which

are like periodic and and um and uh and

event based. And so the future

observability actually looks a lot more

like this than it does clicking around

graphana UI.

So first off evidence well normally

these like or are or what do you start

with what do you look at? Uh traces are

are pretty nice logs as well. uh but but

you know most of the AI systems these

days have like traces at the core of of

the agent framework. So, so you kind of

start with with looking at traces and

this is this could be periodic you know

every five minutes this could be based

upon an event an error and normally

there's some combinations of these which

is um you know some some like uh context

and log you know context and skills used

to put together logs maybe there's the

repo uh you want kind of a combination

of all this together um to understand

what to go fix the repo tells you the

code path that you know the you know all

tells you everything that's there, the

the production logs or traces that the

agent pulls down. Um, normally our

skills actually pull pull little temp

files down into the the repo. Um, so

that you kind of have this this this

idea of what actually happened, what the

code is there enough and and all that

together to put up a fix. Um, so it's

this combination of the right data and

file format in the repo along with your

code in the repo. That's kind of the

magic of this skills which are

composable for the agent to go actually

put a fix. Um, and a lot of this, some

of you, a lot of you probably do this

locally today. You you run this locally.

You have an agent that you you kick up,

maybe you're spinning up, but it's on

your laptop. And I think we all feel

this this this move from from this

laptop um to to maybe to to basically

sandboxes. Um

and and really the sandbox is this this

running environment where um based upon

an event or a periodic you know a

periodic event you can kick this thing

off and it does the same thing you were

doing locally get it working locally

first locally on your laptop and then

event based based upon the observability

platforms like ourselves. Um you can

trigger these on a schedule or or kind

of you know every every error that comes

up. Um, and generally, you know, you

generally it's kind of put putting the

loop together to do this and and um, and

I want to kind of give you one example.

So, this is Alex, our agent. This is a a

real example. It's a very simple one.

And then I'm going to show you what it

looks like in product. Um, this what we

use every day. Um, but this is just an

example where um, we had a stream

canceled event. So um so Alex is is

basically um Alex is is basically our

our inproduct assistant. Um to-do update

is is a uh is a is a way of of managing

kind of its its task list. Um and it was

trying to you know I'll walk you through

the the error in a second but basically

um it's calling a bunch of these two

to-do updates and kind of um errors out

and and so for us it's it's you know how

do I put the data together um to debug

this? how do we do it automatically and

and signal is just something that's

running in the background for us that's

putting up like issues relative to these

things. Um this was uh a kind of one or

two line fix that it comes up with.

These are these are ideal but a lot of

times the fixes are bigger. Um and and

the bigger it is the more likely a

human's involved in kind of like

spearheading it over the line. But again

it's about that that cold start. Can I

start with like all this information on

the issue and guide it the rest of the

way is kind of where we are right now.

Um, and for us, your job kind of moves

from responder to reviewer. Um, and and

and the view is like traces and evals

don't go away in any way, shape, or

form. They're just they're they're a key

part of the loop. Now, you're going to

trace 10 times more. You're going to log

10 times more because that helps you

know what path your software took.

Before, you wouldn't do that because

because humans can't dig through all the

logs. It's just noise. But by logging

and tracing more of your like is it

every inch of your software? Maybe in

some places. Um by logging and tracing

orders and orders of magnitude more than

we do today, we can actually create

these continuous loops that know what

path was taking your software and and

and actually have it fix itself. So this

is kind of my vision for where I think

things are going. um in a way and and

for us I'll show you signal in a second

and you'll see all these I mean I feel

like there's there's this think of this

as an an SR you know something that

helps you debug maybe SR for for AI um

but I feel like there's a lot of black

boxes out there like oh there's a SR

agent that does this or S agent that

does this all we're really trying to do

ourselves is take your local debugging

experience with cloud code cursor and

run it periodically so pick your sandbox

pick your harness, pick your skills,

we'll pre-bank a bunch of things with

you. So, we're just trying to again take

the things we were doing locally and

actually run them um uh you know, run

them in a system. So, we believe in you

know an open approach um to this and um

I I'll give you a demo of what this

looks like um from a product

perspective. So,

so this is um this is a a financial

trading agent. Um given what you saw in

the previous

uh presentation, I would not recommend

doing a financial trading agent. Um they

they they're unlikely to make you money.

Uh at least not not yet. Um maybe

there's some people uh doing it good.

But long story short is this one's, you

know, uh people asking questions about

stock trading right now and it's giving

giving answers. Um there's a lot of ways

this this can fail. And so this this

gives you this is a rise. It's a

platform. So first off from let me

describe the products we have. Uh this

is AX which is our our SAS platform. Um

we also have Phoenix which is open

source if you just want to start

tomorrow. Um Signal right now is is just

available in in our AX SAS platform. Um

which also could be deployed VPC but but

this so so give you an idea of our

product lines. Uh if you want to try out

Signal, it's it's an AX. Um

and what it looks like is something like

this, which is it's just periodically

running and and kind of coming up with

like issues and you can hook it up to

your GitHub repo. It can create an issue

in your repo. You can create an

evaluator from this. Maybe u maybe

there's a a specific problem by which

you want to catch again. You can add

these to a data set. So if you want to

add these and and it has evidence

associated with this like traces um in

this case um in this one it has skills

like for Google cloud and some other

logging systems. So we can frontend a

bunch of places the data we're we're

pretty good at building I think these

skills to debug issues again uh but you

can add your own skills. So these

examples here you know traces running

out without a guardrail. Um there's

there's um uh you know safety safety

issues and intent issues and a lot of

this too is like you know how does this

work? How do I you know it feels a

little too black box to me. Well all

this is open and open box um in the

sense that um I can set up you know I

can set up the harness that I want it to

run on. This one's cloud code. I can

pick my sandboxes and sandbox systems.

Um, I can use cloud managed agents if I

want. I can use Arise sandbox. Uh, why

would I want to use Arise sandboxes

versus cloud managed agents? Well, a lot

of our customers um don't want to

connect their production systems to

Tanthropic. You know, you you you want

your s you want these these sandboxes to

debug your database or connect to it. So

we install in the VPC of a lot of you

know bigname companies out there um from

from Uber to um to bookings to you name

it and and these people don't want to

send their connections out but they'll

they'll use a s you know many many

companies um are very comfortable

installing a V into a VPC and actually

connecting it up. So you can use Arise

sandboxes or you can use Daytona or any

of any of these that you're comfortable

with um that you built relationships

with. Um and and then from a a platform

perspective, you know, we support

um running we support, you know,

tracking the different agents that

you're running. So you have this swarm

of agents. Maybe you've kicked off um

maybe you're kicking off a signal which

which is our agent. It's running

periodically. Maybe you're kicking off

your you know you've named another agent

um in the system. And these all support,

you know, viewing the session that that

ran, downloading the transcript, and you

can resume a clouded session locally,

too. So, the idea is that this thing's

constantly running. You're picking the

harness, the sandbox. You're deciding

the prompt if you want, hey, don't be

aggressive or look for, you know, look

for security issues. So, you're deciding

the prompts that drive this and you're

also deciding the skills that go along

with this. So um in

a preset here um I can add you know add

different skills. I can add my own

skills. I can link repos. I can I have

pre-baked skills too. Um so the ideas

observability platforms are really

starting to get are becoming tied to the

continuous loop to the the fix not just

the the signal. Um, and and you you want

to take your local experience you have

debugging the stuff locally. You want to

take the evals that are running and and

actually have these all work in

something that puts up a fix or at least

gets you a cold start and then I can

take it over locally if I want to

continue debugging uh from from here.

Um, so this gives you um a rough idea of

of kind of of of signal to PR um what

we're doing. Um, I did want to offer,

you know, questions if people people

have any questions on what we're doing

or how we see uh the industry evolving.

Happy to happy to answer. Thank you.

[applause]

>> Yeah. Go ahead.

question of like why can't we just

connect cloud code to your data and have

cloud code do all these things. I think

there's like a version of that question

that can probably be asked for these

autofixes, right? Like why not have

cloud code read the traces and push the

PR itself.

>> I'm curious how you would respond to

that question.

>> Yeah. Uh so so why wouldn't have cloud

code kind of hook to your your data and

just just do it? Um the answer is like

you should um like like the vision and

what we do actually at at Arise is we

have uh a lot of skills. I think first

off to to make that really work well you

have to do a bit of well-designed skills

in the data space. The skill like the

the important things of designing these

skills are are around really around

getting data you know finding the right

data first. So I want to find a group of

traces relative to a session or

something. Getting that data into the

repo in a file format. These harnesses

are magical with files. So you get the

file what happened. In some cases we

have 10meg files like sitting in the

repo. Um so it's designing the skill to

be really really well done with the um

with this data and and then giving

claude enough skills to be composable to

find issues. So the answer is absolutely

yes. like we have Pyroscope skills that

will find memory issues. We have facets

in Pyroscope that the skill knows how to

use. I can cohort by customer to see if

a customer is causing an issue. Um but

but you've got to kind of design the

skill surface area in a way that Claude

can really really work well and and and

it's not just like point Claude at the

data.

>> I see. Thank you.

>> Any other questions?

Anyone else?

>> Oh yeah. Okay. One more.

Thanks for the talk. Um, there was quite

a few mention of eval but you know I'm

looking at the traces so you know I

understand the concept of traces but

where where did the evals come in when

you have that signal that says hey

something broke in production.

>> Yeah. So so the so the eval typically

will the um the eval essentially are

running and being layered on typically

to the production traces something we

call online evals. Um let me see if this

one has an example here of it. Um so so

eval actually are data on the trace

itself and so the agent knows how to uh

grab the data from traces knows how to

um visualize and you know the skills to

basically pull data for for the

aggregate values of evals across the

traces so that so the the skills that

you give uh the harness allow it to get

the data on the eval from from the

traces. Um so eval are kind of like I

view them as le at least a first

generation eval evals which are elements

a judge um as a as a AI layer that

allows you to run periodically and and

assess your system but it's like it but

it's adding a little bit more you know

pre-processed information on the data

that that and then as signal is running

it's using data from the evals that were

layered on um in addition to all the raw

data that it has there Um it but it

tends to be like you build an eval for a

failure you've seen before a lot of

times. So I have these prompt injection

things that I'm trying to catch or

something or um or or a failure in the

way it's responded maybe to to something

before. So they they tend to be this

like you know u at least the LM as a

judge is tends to be like this this

thing you um preset up and then you can

actually create evaluators for failure.

Say you find this failure that's pretty

common and happening all the time. I can

create an eval so I can catch it next

time. I just you think of it as like

almost a an AI um assessment that's

always running. Uh the other note is the

element as a judge can run really at

scale. Well, every you know I have

customers who who lay you know layer

element as a judge across um their full

data set uh where where this tends to be

like you know uh more periodic on a lot

of data. So cool. Thank you.