Cory Forsyth

The willSetProperty gotcha in Ember-Data: Understanding the state machine

cory — Mon, 10 Jun 2013 20:31:31 +0000

Ember Data, the ambitious comprehensive persistence library solution for Ember.js, has been growing in popularity, and even though it still has a prominent disclaimer that it is not production-ready, it has recently been included in the models section of the official ember.js guides. This seems like a good time to start familiarizing ourselves with it.

Summary

I'm going to describe a particular gotcha that I experienced while using ember-data, and then explore through the ember-data code and documentation to try to piece together what happened and how to avoid it. Short version: If you attempt to change a DS.Model's attribute while it is inFlight, ember will give an error. There's an interactive visual app on jsbin that you can use to explore this yourself. If people find this useful I will write a follow-up that shows an example usage of Kris Selden's Buffered Proxy as a way to mitigate this issue. Follow me on twitter to get notified when part two is up.

DS.State Events, Flags and Transitions

The error that I kept encountering was this one, that would seemingly randomly show up in my javascript console:

Attempted to handle event `willSetProperty` on
 while in state
rootState.loaded.updated.inFlight.
Called with {reference: [object Object],
store: , name: title}

There are two important parts of this error message: willSetProperty, and the current state: rootState.loaded.updated.inFlight. What the error is trying to tell us is that the model's was in the rootState.loaded.updated.inFlight state when something caused ember-data to try to set a property ('title') on the model. We get the error because the model's state machine tried to send the event 'willSetProperty' but the state machine's current state (rootState.loaded.updated.inFlight) does not define that event.

So let's back up bit more and ask: What does it mean for the record to be in the `rootState.loaded.updated.inFlight` state? How did it get that way? To answer this we need to look a bit at the state machine that ember-data uses internally with its models. That code is in the 'system/model/states.js' package of ember-data, and in the comment it gives a good explanation of how the state manager works. Quoth the docs:

A record's state manager explicitly tracks what state a record is in at any given time.

The class is `DS.State`, which is a subclass of Ember's `Ember.State`. At any rate, the important thing you need to know for our purposes about a `DS.State` is that it has three different components that make it up: Events, Flags, and Transitions.

DS.State Events, Flags and Transitions

Flags are boolean values that are meant to describe the model's current state. `isLoaded`, `isSaving`, etc are flags. The ember.js model lifecycle guide lists all the flags.

Events are, again quoting the ember-data comments: "named functions that are invoked when sent to a record. The state manager will first look for a method with the given name on the current state." States can be nested and so when attempting to respond to an event they will look up their class hierarchy at the parent, grandparent, etc. until they find an event with that name. If they don't find one, they raise an error (we'll come back to that!). It's also worth nothing that events are invoked on the record (using the method `send`), which then delegates to its internal state manager. So you do something like `myPost.send('myEvent')`, which calls `myEvent` on the post's state manager.

Ember-Data states can be `transition`-ed between using `transitionTo`. Transitions work similarly to events but they are intended to be invoked when handling an event, never directly. So a typical usage pattern might be to call `myPost.send('gotFoo')`, which would delegate to the state manager, which in turn would do some internal bookkeeping and then, perhaps, transition to a new state called 'hasFoo' using the call `transitionTo('hasFoo')`.

So with that in mind we can break down our error message. Something tried to invoke the event `willSetProperty` on our instance of `App.Post`. The App.Post was in the state `rootState.loaded.updated.inFlight`, and that `inFlight` state does not have an event named `willSetProperty`. The code for the `inFlight` `DS.State` is on github. If you look at the code you can see that the events on the `inFlight` state are: ` materializingData`, `didCommit`, `didChangeData`, `becameInvalid`, and `becameError`. There is no `willSetProperty` event, hence our error.

How does a model get into `inFlight` and why might its properties change therein?

We are getting pretty close to unraveling this error. We now know what it means. The next step is to figure out what caused it. How did our model get into `inFlight` state and why did ember-data try to change its `title` property while it was in that state?

The answer to the first question is pretty simple to answer, and in fact if we look back at the comments above the `inFlight` `DS.State` it will not only tell us how a model gets into this state but warn us that changing the model's properties should not be done during that time. The comments in the code say:

Once a record has been handed off to the adapter to be saved, it is in the 'in flight' state. Changes to the record cannot be made during this window.

So our App.Post was set to `inFlight` when we asked ember-data to save it to the backend, perhaps by calling `model.save()`. That makes sense. The post will be in the `inFlight` state from the time we call `save()` until the time that the backend api responds to ember-data's request, at which time ember-data knows that the changes have been persisted. At that point it will transition to the `saved` state. Internally, ember-data's `inFlight` state transitions by calling `manager.transitionTo('saved');`.

The last part of this puzzle is to find out why the post's `title` property was changed between the time we started saving the post and the backend api responded. There are actually a number of reasons this might happen, depending on the nature of your application. For instance, if you have an `Ember.TextField` whose value is bound to the post's title property, then if you allow the user to type in that field while the post is being saved you'll see this error. Likewise, if you have other async processes going on that could modify the post, then you need to make sure they don't do so when the post is in flight. In my specific case, the post was listening for another user's updates via websockets, and the callback that responded to the websocket message was attempting to update the post's title without ensuring that the post wasn't in the middle of saving.

Demo!

To see visually how the state and state flags change when saving an ember-data model, check out this demo app on jsbin.

Conference Strategies and GORUCO 2013 reflections

cory — Mon, 10 Jun 2013 19:44:36 +0000

This post is a reflection GORUCO 2013, and at the end I give some advice on how to make the most out of a conference.

I have been a volunteer co-organizer of GORUCO (aka Gotham Ruby Conference) for 5 years in a row now. It's been quite a ride, and every year it's a good checkpoint experience. On a personal level, comparing this year to years past helps me to see my own evolution as a Rubyist and a member of this community. At the first GORUCO I attended, 6 years ago, I remember knowing almost no one and feeling that lots of the talks were well over my head. I was a ruby noob and the experience was intimidating. It's encouraging and rewarding to reflect now and see that over the years I've known more people and felt more comfortable every year. I have similar thoughts when looking at the conference as a whole. Every year it has gotten bigger, and, in my opinion, better. That's hard to do.

Photo from @_eirik.

One of the things I am particularly proud of this year is the scholarship program we put together with RubyCentral. They had experimented at Mountain West Ruby Conf with paying the way for certain rubyists to attend and pairing them with mentors at the conference (their term was "sherpa"). We were able to continue that program at GORUCO this year. RubyCentral put up the money for 10 tickets that we made available to students who would otherwise have had trouble affording the conference. Our ticket price this year was the highest it has been, and the organizers put a lot of work into mitigating that by making it easier for under-represented groups to attend. This scholarship program was part of that effort. You can read more about the program on GORUCO's blog.

A real credit to the NYC Ruby community is the ease with which we were able to find volunteer mentors, who spent the day of the conference (and also by and large met up pre-conference) with their apprentices/mentees to help them get the most out of the conference. I actually had to turn away a number of volunteer mentors because we ended up with too many. It really is great to see how willing people in this community are to give their own time to help newcomers.

I met a few of the scholarship recipients during the day and it reminded me of myself at my first GORUCO 6 years ago. Having a mentor back then would have gone a long way in making the conference a better experience for me, so I'm happy that we were able to do that for the recipients this year.

At one point I spoke with Sandi Metz, a real Ruby luminary (her book, Practical Object-Oriented Design in Ruby, is the number one Ruby book I would recommend to anyone) and also a mentor, about how things were going with her apprentice. We got to talking about general good strategies for conference attendees. Whether you are at your first conference or your fiftieth, what can you do to make the most out of it? Some ideas:

Approach people. At a conference I often feel like I did in middle school. Lots of cliques of people talking all around me and no way to break in. It can be hard to do, but striding forward up to a person or group of people and saying hi and introducing yourself is one of the best things you can do at a conference. A conference is a great opportunity to meet a bunch of people in your community in the flesh. If you didn't take advantage of that you might as well just read a bunch of blog posts or watch the videos online later.
Be approachable. Like I just said, conferences are made up of people. Each of us being there contributes to the conference for everyone else. The better the contribution you can make, the better the conference can be for everyone. Being inclusive goes a long way, and makes point number one above work even better. When someone sidles up to your group, take advantage of a pause in the conversation to bring them into the conversation with a quick introduction. Just turn and introduce yourself, or if you already know them, proactively intro them to the other people nearby and summarize the conversation so far.
Be ready to learn. The common thread in all these bullet points is: Do the things at a conference that you can only do at a conference. Pay attention to the talks. I know that not all talks are interesting to all people. Some talks can be pretty boring. But still, persevere to avoid using your computer. You can write emails and browse Hacker News anytime; you can only be an attendee at GORUCO once a year. I sometimes keep a laptop open so that I can do complementary research on the speaker's subject while they are talking, or take notes. I think these are worthwhile uses of the computer during a talk, but I still find myself getting distracted and I try to avoid it as much as I can. The best speakers are so dynamic that you can't help but stay focused on them, but for those that aren't, try to follow along anyway. You never know when they'll say something that you can really learn from.The wifi will invariably be spotty anyways, and the internet can wait.

Finally, I wanted to say thank you to: the other organizers. They are great to work with and I'm happy that I've been able to be included; the NYC Ruby community: it's a great community to be a part of, and you are the critical component in making GORUCO a success every year.

<3

Programmatically List Routes/Paths from inside your Rails App

cory — Mon, 03 Jun 2013 00:39:42 +0000

This post explains how to write code that can introspect a Rails app at runtime and determine what routes have been defined. Similar to `rake routes`, but something that you can use from inside your rails app.

(If you would like to skip ahead and just see the code, here is the gist.)

Why would you want to do this? Say you are creating a new rails site. You start with a home page and as the site gets larger you add other top-level paths like "/about" and "/team". This app is a fancy new color-picker and so you add a whimsically-named page "/rainbow", where you educate your users about light and color. Some time passes, and you decide you want to add a top-level page for every user at "/:username". Things hum along and then you launch the site. One day a user signs up with her preferred username "rainbow".

Now your site has two "/rainbow" pages: one is your static educational page. The other is this new user's page. This is a problem. If you were able to introspect your app's routes at the time that you validate a new user, you could have noticed that the requested username conflicted with an existing path and prevented the creation of that user.

Let's figure out how to do this. We'll start by looking at the source code for the rake routes task. The important line is this one:

all_routes = Rails.application.routes.routes

Let's inspect this code on the rails console to get a better idea of what we are looking at here.

>> Rails.application.routes.class
ActionDispatch::Routing::RouteSet
>> Rails.application.routes.routes.class
Journey::Routes

So our variable `all_routes` is an instance of a `Journey::Routes` class. What's journey? Journey is the router that Rails uses (and unfortunately its README is a bit terse, so there's not much more I can say about that). If we look at the code for Journey::Routes we see some enumerator-type methods (`each`, `last`, `length`). This is meant to be iterated over, once for each route. (If you look carefully you'll see that it also includes an `attr_reader` named `:routes`, so we could get really crazy here and access the routes as an array using: `Rails.application.routes.routes.routes`. But that's just painful). When we inspect each element of the `Journey::Routes` object, we see that each one is, as expected, an instance of `Journey::Route`.

Now we know how to get the routes, using the same code that `rake routes` does. Next step is to figure out how to determine the actual path that a `Journey::Route` object represents.

At this point, looking at the code in Journey isn't all that helpful. There are a lot of avenues we might explore here. Each `Journey::Route` has a number of potentially-promising attributes and methods, such as `path`, `name`, `segments` and `parts`, but each one just takes us deeper down the rabbit hole and the path (no pun intended) from here to a string representing this path is still elusive. Let's look again at the `rake routes` task, since we know that this task does indeed turn the routes to strings to display on the console.

Looking at the rake task again, we see the following lines, culminating in one that `puts`-es the routes to the console:

  require 'action_dispatch/routing/inspector'
  inspector = ActionDispatch::Routing::RoutesInspector.new(all_routes)
  puts inspector.format(ActionDispatch::Routing::ConsoleFormatter.new, ENV['CONTROLLER'])

So let's look at `ActionDispatch::Routing::RoutesInspector`. Hmm, this `path` method looks promising:

  def path
    super.spec.to_s
  end

It's not entirely clear here what `super` refers to since we are looking at the `RouteWrapper` class here and its parent is a SimpleDelegator, but it is reasonable to expect that its parent delegates to a `Journey::Route`. So let's go back to the rails console and try something out with that in mind. The call to `super` from `RouteWrapper#path` simply becomes a call to `path` on the `Journey::Route`:

>> routes_set = Rails.application.routes.routes
...
>> first_route = routes_set.first
...
>> puts first_route.path.spec.to_s
"/assets"

Bingo! You can try this in your own rails app:

>> Rails.application.routes.routes.each do |route|
>>    puts route.path.spec.to_s
>> end;nil

Great! Now all that's left to do is to iterate through those routes and extract the initial path segment from each one. Here is the code:

To use it within your app, do this:

top_level_paths = \
  RouteRecognizer.new.initial_path_segments

A short lesson in priorities, courtesy of the MTA

cory — Mon, 27 May 2013 15:20:54 +0000

If you are one of the millions of of people who ride the MTA's Metro North trains every year, you've probably bought a ticket at one of its ticket machines. After choosing your station and other information, the screen prints your tickets and they drop into the metal tray at the bottom for you to collect.

While the tickets are printing, the screen displays this message - "Your ticket(s) are being printed."

This little hedge strikes me every time. An organization as large as the MTA, with an annual user base (ridership) in the millions (76.9 million in 2006), with a software/hardware project that, presumably, was contracted out for millions of dollars, and no one felt it important enough to get this part of the experience — a part that *every* customer experiences — perfect. No one put in a spec somewhere that the message must be "Your ticket is being printed" if it is one ticket and "Your tickets are being printed" otherwise.

Millions of dollars. Millions of users. And: "Your ticket(s) are being printed."

It's a great lesson in stopping at "good enough." I've worked with many startups that get hung up on small details like this. When perfection is achievable it's easy to feel that it should be a priority. But if an organization like the MTA with millions of users and millions of dollars can live with "Your ticket(s) are being printed," maybe your startup can live with "good enough," too.

Thoughts on New Year's Resolutions

cory — Mon, 31 Dec 2012 19:03:59 +0000

Like most people, I have historically sucked at keeping my new year's resolutions. I used to be heavily goal-oriented around late December and (when I'm procrastinating -- bad start already!) early January, but recently my thinking on goals and goal-setting has, as a politician would say, evolved. I'm approaching this a little bit differently this year. Here are some of my thoughts on how to make good goals and, if you're still inclined after reading to the end, new year's resolutions as well.

For some context and an alternative viewpoint, skim Leo Babauta's "No Goals" post. Then, if you have time, watch him debate goals with Tim Ferris for some counter-points.

Until a few weeks ago, I wasn't planning on making any resolutions at all this year. I was down with the No Goals manifesto. But I've changed my thinking and instead I am going to work on introducing some small habits and align them with the goals that those habits reinforce.

Make SMART Goals

See Paul J. Meyer's SMART goals definitions. It's an acronym, and the letters stand for Specific, Measurable, Attainable (within your abilities to achieve), Relevant (this the least important component, imo) and Time-Bound.

For example, "Lose weight" is a bad goal. How much weight? That's not specific, and it's not time-bound.

"Lose 10 pounds": Better, but not good enough yet. There's no time-limit.

"Lose 10 pounds by the end of January." Ok, now we are talking. The best goal is one where a friend of yours can objectively say whether or not you succeeded. In this case they'd need to see you stand on a scale but you get the idea -- if it's not something that an external person can say that you've completed, the goal is too ambiguous or subjective. If your friend can see the scale and it shows 10 fewer pounds than when you started, and it's not February yet, they can give you a big high five and say that you are #winning.

Use short time frames. AKA ABR (Always Be Resolving)

The thing I dislike most about the new year's resolution concept is the idea of making a resolution for a whole year. That's a long time! Humans on the whole are bad at knowing what we want today and worse at predicting what we'll want tomorrow -- forget about predicting what your priorities will be a year from now. We also suck at knowing how difficult it is to simply do one small action (such as ordering a salad instead of a burger) day in and day out. It's hard and your motivation waxes and wanes over time -- sticking even with something easy is quite difficult when you extrapolate that over a large period of time.

Long time frames are bad for another reason common to all humans: We procrastinate. In reality a 12-month deadline mostly just gives you about 11 months to not really think about your goal, by which time it will be next December and the goal's urgency will seem far away, and you'll be too busy with another holiday season to do anything about it.

For this reason, I think the maximum time frame that makes sense for any goal is a month. This doesn't mean you shouldn't make loftier goals. Just break them down into human-sized timescale components. For example, if your goal is to do 100 push-ups in a row in a year, break that down into a goal of 30 push-ups by the end of this month, 40 by the end of next month, and so on.

Because we are bad at knowing what we really want and whether it will still be relevant in the future, don't over-commit. I recently read Steve Pavlina's Personal Development for Smart People: The Conscious Pursuit of Personal Growth, and he is a big proponent of doing 30-day experiments, the idea being to try out a new goal or behavior for only 30 days and see how you feel at the end of that trial. If it's something that is a good fit for your life you will naturally stick with it, and if it isn't you can freely drop that goal after the trial.

I like this approach because another thing I dislike about typical new year's resolutions is their implication that you are committing yourself to sticking with whatever goal it is for the entire year. As I said above we're pretty bad at understanding how difficult that will be and how important that goal will be to us in a year's time. Not to mention that committing to doing something for an entire year is daunting (and psychologically quite intimidating). Steve's 30-day trial idea is a good compromise.

Update: I just randomly came across a Ted talk by Google Engineer Matt Cutts extolling 30-day trials.

And really what you want to do anyway is Always Be Resolving -- in other words don't let this one time of the year be the only time you stop to think about what you want. Make that a part of your daily or weekly rhythm, to ask yourself whether you are getting closer to whatever goals are important to you. And then freely course-correct as you go along. Don't stick with things that aren't working or that are no longer important to you, and reinforce/tweak goals that are working for you.

Recognize that Goals are reinforced by Habits

Goals are the finish lines on the horizon, but daily and weekly habits are the footsteps needed to get there. You achieve a goal by doing something (or avoiding the doing of something) on a mostly-daily basis between now and the end of your goal. Something you do every day like this is a habit. So when making a goal, think of what habit(s) you'll need to pick up in order to achieve that goal, and plan accordingly.

Creating (positive) habits is far more difficult than most people realize. BJ Fogg at Stanford has a free one-week program called Tiny Habits that will get you thinking in more detail about habits than you probably ever have before. I highly recommend it. Learning how to create and grow your habits is key to making bigger changes in your life and achieving your goals.

Hold Yourself Accountable

Good goals are measurable, so measure your progress frequently to ensure you're moving in the right direction. BeeMinder is a good website to check out for this.

For added motivation, state your goal publicly. Tell friends and family and encourage them to check up on you. Update: Derek Sivers disagrees in this short Ted talk. (My take: The research is flawed. I still think publicly stating your goals is a good idea.)

I'll take my own medicine and state a current goal here:

For the thirty days ending January 29th, 2013, I am going to try being a vegetarian. I'll still eat dairy and eggs, but no meat or fish. I want to see how difficult it is and whether I feel any differently at the end.

Be Gentle on Yourself

Improving oneself is hard stuff. Motivation has valleys in addition to peaks, and from time to time you will fail to keep up with your goals. This is normal and not really a bad thing. You should be making goals that are hard to reach, which means you will fail to achieve them sometimes. That's ok. Rather than getting frustrated, use those failures as a learning opportunity -- figure out what didn't work, adjust the goal, and try again.

If you've ever tried meditating, one of the things you'll hear is that when your mind inevitably gets distracted and you realize you have lost your focus, don't dwell on your loss of focus. Instead, gently acknowledge that your mind wandered and try again. Habits and goals have a similar sort of zen: Keep trying, don't be too hard on yourself when they don't work out, and over time you'll enjoy the journey as much as the destination.

How Target Sells More By Showing You Ads For Things It Knows You Won't Buy

cory — Tue, 30 Oct 2012 22:31:55 +0000

Lots of great stuff in Charles Duhigg's NYT article about consumer data mining and habits, but one of the things that stuck out the most to me is how Target will deliberately insert ads into a weekly circular for items it knows the intended recipients have no interest in buying. And this allows them to sell more!

"Then we started mixing in all these ads for things we knew pregnant women would never buy, so the baby ads looked random. We’d put an ad for a lawn mower next to diapers. We’d put a coupon for wineglasses next to infant clothes. That way, it looked like all the products were chosen by chance. And we found out that as long as a pregnant woman thinks she hasn’t been spied on, she’ll use the coupons. She just assumes that everyone else on her block got the same mailer for diapers and cribs. As long as we don’t spook her, it works."

There's so much interesting stuff about the irrationality of humans that we're only starting to unpack now.

On-boarding New Users (Class Notes)

cory — Thu, 14 Jun 2012 15:50:11 +0000

Saw a good class at Ben Chow about on-boarding new users. My unfiltered notes:

Understanding: It's not enough to just explain your product. You have to 'take their hands' and show them
Tips:
1 grab hands and guide them
-- big arrow pointing to what they should do
-- Progress bar
2 sell your value proposition
3 get personal -- give personalized info/results as fast as possible
4 invite friends and make it make sense (give users a reason to invite)
5 help set up
6 measure performance, optimize funnel

What do you want the user to do every time they use the product? The users want the same thing!

Tell ppl what you do, how you do it, and tell the how they will use it. Think of all the reasons someone would say no and not use it, and preemptively address those

the thing we want to do is build habit. Get ppl coming back frequently. And every time they come back, show them something awesome

Videos do a great job informing but they are passive. Users need to start clicking and doing to build those habits. Not a big believer in using videos in the on-boarding flow because they take a lot of time. The user should move very quickly. Keep them moving and doing.

5-second rule: you have the users attention for 5 seconds. Give them something interesting to do/click every 5 seconds while on-boarding...each time they click, they are giving you another 5 seconds of their attention span.

enemies:
Confusion, apathy, and time
If they're confused, if they're bored, if they are not getting something quick, they're gone
Most users who are trying your app want to love it. So give them something to love.

"how does this product fit in my life?" contextualize for your users. Explain how you fit, how best they can use, etc

Get contact info as soon as possible. Eg get their email before their CC. That way if you lose someone before they complete a purchase you can contact them and offer a free trial or otherwise get another shot at converting them

Email marketing: back off the frequency every time they don't click on your email, otherwise you risk them unsubscribing.

CTO School: Skillshare Methodology and Resets with Art.sy's CTO

cory — Tue, 15 May 2012 01:38:32 +0000

Always a good time at CTO School. This week was Product-Focused Processes and Resets. I am feeling more and more at these that there's a big divergence in what being a CTO means, and the divide is primarily between big (bureaucratic, politically-fraught) organizations and startups (experimentation, continuous improvement, well-greased wheels). At this and the last one I went to, one of the talks felt like it was preaching to my personal choir, the other felt like it was above my pay grade and beyond my bureaucratic tolerance. Even so, I have found them all consistently interesting. Here are my notes:

Cto school notes may 14

/Cto school notes may 14

Malcom Casey: @malcolmcasey from Skillshare

Process
- Align product vision and goals
  - Everyone on team is involved
  - Not top down
  - Product planning and prioritization
    - Identify top problems the product is facing or the users are facing
    - 2 weeks
    - Ask why 5 times
    - What does successs look like, how can we measure thay
  - Product sprint
    - Identify owners and milestones
      - This can be a junior person.
      - Doesn't have to be an expert
      - The owner is in charge of just keeping everything on track, not necessarily implementing all of it
      - They call the owners 'zookeepers'
    - Don't micromanage
    - Strategy
    - Product design
    - Implementation & deployment
    - Verification!
      - Make sure after releasing that the hypotheses were validated
      - Decide whether to keep or kill it
    - Meetings
      - Keep them focused
      - Use hip chat/email/etc, get the feedback
      - As soon as they are ready to make a decision, then schedule the meeting
      - Keep the meeting to 15-30 mins
  - Product design
    - Sketching
      - Everyone on team would sketch, come up with a few solutions
      - 3/1 process (cf apple)
      - Come up w 10 solutions, pick the best 3, refine that, pick the best 1
      - After everyone has done this, agreed, likes it, the move on to wireframes, flows, etc
    - Wireframes
    - User stories & flows
    - Product design != ux/ui design
- Never "launching" always shipping
Tools
- Product design
  - Ux/ui.
- File mgmt
  - Dropbox
- Product mgmt
  - Trello
  - Google docs
  - Used to use pivotal tracker but it didn't work for then
- Bug tracking
  - Lighthouse
- To do list
  - Behance action runner
  - Asana
Culture principals
- PayPal mafia (graphic showing how all the early founders went on to start new successes)
- It's not about making ideas, it's about making ideas happen
- Creating great team
  - Not about rock stars, about finding a team that meshes well.
- Over communicate
- Transparency
  - Tell employees about vc talks, where the company is going, etc
- Create solutions not problems. Think about how to fix, mot just pointing out problems
- Ship it
  - Done is better than perfect
- Startups within startups
- Set unrealistic deadlines
  - Estimate what you can do, then aim for 1.5x that
- All new hires sign an agreement that they understand and agree with the culture
- Simplify and enhance focus
Focus
- Finding the most important thing to work on is critical
- Startups don't starve they drown (in ideas)
- Focus means saying no to distractions, not saying yes to the one thing (Steve jobs, paraphrased)
Questions
- How to get focused?
  - What experiments do you want to run, what can get done in 2 weeks
- How many hats do ppl wear?
  - 3-4. Helping creative process, (developing), analytics driven (a/b testing), teaching everyone SQL to look at DB, recruiting
- Does all the shipping cause bad bugs to go live?
  - Baseline testing. Err on shipping first, fixing later
  - They have had critical bugs but they fix quickly, they keep users happy
- What's the turnover?
  - Haven't had anyone quit. Everyone who has left has been the company's decision
  - Very transparent, strong culture
- What's the hiring process?
  - Mostly referrals
  - Don't use recruiters
  - Inbounds/branding/team branding
  - Etsy does a good job of this and is an inspiration
  - Very thorough process, multiple phone interviews, homework, meet everyone, check references, keep it within 3 weeks
  - Try to hire for where you want the team to be 12 months from now
- How many have you let go?
  - About 5 in a year
- How do you find the balance of too much and too little process?
  - No right answer.
  - Play around with it
- Any special incentives to keep ppl engaged/motivated?
  - Allow ppl to take as many classes
  - Allow ppl to go in groups of up to 4 and spend company money to do anything they want
  - Give lots of freedom and responsibility

CTO Artsy Daniel Doubrovkine: Pressing the big reset button (@dblockdotorg. Slides).

How do we feel about resets?
- It's negative, happens as a consequence of failure
- Always happens with drama. 'slam the breaks'
- Compared to changing the wings on a flying airplane
Examples
- Exhibit 1 Microsoft windows vista/longhorn
  - Dev began in 2001
  - Very feature-bloated. By 2004 clearly going nowhere
  - Reset in 2004 on top of windows server 2003
  - In 2006 it was released as vista (major failure)
- Exhibit 2 Microsoft office 98 & netdocs
  - Netdocs was created by microsofties who thought the Internet would kill office
  - Net docs 1 was java, effectively an office reset
  - Net docs 2 reset in C++
  - Net docs 3 reset as a service
  - Spent $500M over the years, 500 devs
Why do resets happen? Survey of about 2000 reset projects. Data from CTO club in NYC and personal network
- 4/10: sunk cost kept them from resetting
- 100% say due to some type of failure
- 7/10 faced strong opposition to resetting
- 2/10 blame pure product failure
- 3/10 blame pure tech failure
- 2/10 blame ppl failure
- 3/10 resets led to success
- 4/10 led to failure
- 100% of projects that weren't reset failed
When is it right to reset?
- always, provided you are 100% sure the project is failing (Ed: tautology?)
- Find truth-Sayers, trusted ppl who will be honest & help you know if it is really failing
- Measure how much cost sunk & how much value it is providing
  - Questioner points out: this is the sunk cost fallacy, better to focus on the value
- Don't be a chicken
  - (chicken & pig committed v involved story)
  - Only those that are committed should decide
- It can be a smart career move to make the hard choice
- Can be very dumb career move if you fail or the organization doesn't support you in the reset after you hit the button
How to reset
- Act now, decisively
- Invest in trust capital
  - Have to compromise in order to gain trust. For every year of trust capital you have a about a month to burn when in a reset
  - Build it by helping others achieve their goals
- Double or nothing
- Split execute
- Under promise & over deliver
  - Audience points out: this is the exact opposite of Malcolm Casey's strategy
Aftermath of a reset
- People forget very quickly how bad it was
- Don't expe t to be glorified afterward. This is just part of your job
- Making decisions about when/where to continue dev paths is always part of your job
- Foster a culture of frequent but smaller resets
  - Experiment often
  - If things are posed as experiments, and you let devs try their ideas, you will keep evolving (Ed: and they will be happy)
- Stop at the top
  - When you're at the top, don't get comfortable...try resetting then to make it even better
    - Ed: wha?!
Questions
- How is the data calibrated?
  - (audience grumbling/saying 'let him speak')
  - Somewhat anecdotal
- How does that affect the code base?
  - Depends, not necessarily software/code (e.g., could be purchased CRM software, when to throw in towel?)
- When ppl from the outside advocate reset, it's not good. Is that a personal opinion or from the data?
- How do you connect the concept of reset to tech debt?
  - They are well connected. I don't like working in environments where you need to explicitly pull out tech debt. That should go without saying
- I pivot on market more than on technology. When you pivoted/reset the tech/rewrote did you get the founders to buy in too?
  - Yes, that is critical, also critical to show that it is promising quickly

Lean Startup Metrics class notes

cory — Mon, 07 May 2012 02:09:45 +0000

I went to the Lean Startup Metrics class on Friday. My notes:

Why metrics important. "what gets measured gets done" Avoid biases in judgment. Numbers don't lie.

5 types: vanity, actionable, predictive, cohort, funnel

Vanity: impressive but don't really mean anything to your business. Vanity metrics are usually aggregate. Funding is too. It doesn't directly relate to success (bc funded startups fail all the time)

Actionable. When you measure it you know what to do. Measure the outcomes and see if your hypothesis is holding true. You have to agree ahead of time what metric to base success on. Cost per acquisition. Find that out early. Big data: FB learned that when women update their profile photo men spend more time on the site. And women are more likely to change theirs when they see other women change. You can use this to get men spending more time on site. Actionable metrics are per-person metrics.

Predictive metrics: Value: What we are doing adds value for customers. Net promoter score: "how likely are you to recommend this?" 1-10. 8-10 are promoters, 1-6 are detractors. Promoter - detractor / total. This is the nps. This is something that you can compare cross industries and products with.

Must-have metric: how disappointed would you be if you can't use this product? You want to see 40% of your users very disappointed. (determined by Sean Ellis) This helps you see if you have something unique.

FB growth metric: w/in 4 weeks of launching a campus, got 90% of the users FB value metric: missed it

Groupon: they started out as a WP blog selling 2-1 pizza coupons.

Groupon's growth model: paid. Their customer is the business, and they have salespeople to get more businesses to sign up.

Funnel metrics: They measure the life cycle of your user. AARRR acquisition activation retention referral revenue Measure how customers move through these stages Determine weaknesses.

Cohort metrics Segmenting user based on when they joined. This helps smooth out seasonal/temporary metrics. Kissmetrics. Mixpanel. Most startups will have to build custom solutions to capture their metrics, because each startup's important metrics are different.

Intro to Lean Customer Development Class Notes

cory — Sat, 14 Apr 2012 15:14:29 +0000

I went to my first Skillshare class last night, Intro to Lean Startup and Customer Development. Very interesting stuff. Here are my notes:

Lean Cust Dev Class

Two biz purposes: mktg & innovation

Find customer for product before you build anything.

Lean startup is: pre-marketing

Startup def: institution designed to deliver a product under conditions of extreme uncertainty.

Startup is all about experimentation and uncertainty.

How do you know what the next step is?

Myth: lean == cheap Lean is about efficiency, investing just enough to validate assumptions.

Myth: lean is only for tech. Not so, it's just about experimentation.

Lean startup story: grocery delivering where they manually delivered groceries to their alpha customers by hand before building any tech to support, just to make sure that they really were delivering what customers wanted.

Why are founders reluctant to ask people about what they think about the idea? One reason: they are afraid of someone stealing the idea. Not so, ideas are cheap, execution is the hard part. Validate the assumption!

Myth: lean replaces vision with data and customer's wants. Vision is very important but your strategy (I.e. solution) needs to be responsive and validated.

UX how to ask questions do experiments to get useful feedback: First step: is there a problem worth solving? The riskiest part of the biz is rarely the solution. Don't say: 'here's a solution, do you like it?' They will say yes for the wrong reasons. How do you ask Q's that explore their experience? "here is a scenario. Have you experienced this? what did you do in this situation?" How did they actually solve the problem? Ask behavioral questions. Need to learn if there is a problem worth solving. The word problem is not always the perfect word (in the development of a game, for example).

Scientific method. Try to disprove hypotheses.

Startups that succeed are the ones that iterate enough before they run out of resources. Failure is not bad, it is part of the learning process. Learn from each failure and use that knowledge to pivot to a new iteration.

Surveys don't give a lot of info. Qualitative interviews give tons of info.

Caveat: in rapid iterations your objective is efficiency in finding product market fit. It's possible to have small enough selections of people that you don't fully validate your assumption (ie you get a false negative).

Good falsifiable hypothesis: [repeatable action] will produce [measurable outcome]

When you ask customers questions, you can't rely on what they say. When people say they will buy X, they will not. (they believe they or someone will, but people don't honestly know themselves that well and aren't that good at predicting their own future behavior). Cf Dan Gilbert quote

How do you decide the next step?

Entrepreneurs are too attached to their ideas. Until they give it a chance to see if it will fail, they can't know if it will succeed.

When you design experiment, start w the single most critical idea. What will break my biz if it is not true? What is most critical and most unknown?

"get out of the building" and work on validating your ideas.

We humans are really good at projecting and experimenting in our heads but bad at predicting others' behavior.

Customers don't care about your solution, they care about their problems. -- Dave McClure

Your obj is to find an early adopter, not a mainstream consumer.

The early adopter: a) has the prob you want to solve and b) they are aware of it. [And bonus if c) they already are trying to hack some solution to it]

You want to find them and build them an MVP.

When you have a lot of uncertainty, it doesn't take much more data to reduce it significantly.

Surveys assume you know the right questions to ask. -Ash Maurya

Myth: Customers don't know what they want. They *do* but they probably can't articulate it well. So don't ask them what they want, ask what they do (in situations like the problem you are trying to solve).

MVP are experiments meant to invalidate your riskiest assumption. Important not to focus on the word 'product'. It's not necessarily a product.

MVP phase 1: talk to people find out how they've solved this problem, see if it's really a problem. MVP 2: product pitch. Can be as simple as paper. Show it to someone and ask "would you use this?"

MVP 3: 'concierge' build something that is just barely enough to be usable, probably w lots of manual help Groupon is a good example. They just emailed ppl, printed coupons and sent them to them, to see if ppl would like this. Didn't build any tech. Tested it w PDFs and email. 'wizard of oz': build a frontend but no tech backend. Example early Charles Schwab wanted to see if ppl would do online trading so they built a frontend for ppl to enter orders but on the backend it was human traders seeing those orders and executing them.

Qs: How to find best ppl to do interviews? Come up w 'persona', find ppl who fits that persona. One type of pivot is 'customer-segment' pivot which is changing to a different customer persona and doing more interviews.