Cto school notes may 14
/Cto school notes may 14
Malcom Casey: @malcolmcasey from Skillshare
- Process
- Align product vision and goals
- Everyone on team is involved
- Not top down
- Product planning and prioritization
- Identify top problems the product is facing or the users are facing
- 2 weeks
- Ask why 5 times
- What does successs look like, how can we measure thay
- Product sprint
- Identify owners and milestones
- This can be a junior person.
- Doesn't have to be an expert
- The owner is in charge of just keeping everything on track, not necessarily implementing all of it
- They call the owners 'zookeepers'
- Don't micromanage
- Strategy
- Product design
- Implementation & deployment
- Verification!
- Make sure after releasing that the hypotheses were validated
- Decide whether to keep or kill it
- Meetings
- Keep them focused
- Use hip chat/email/etc, get the feedback
- As soon as they are ready to make a decision, then schedule the meeting
- Keep the meeting to 15-30 mins
- Identify owners and milestones
- Product design
- Sketching
- Everyone on team would sketch, come up with a few solutions
- 3/1 process (cf apple)
- Come up w 10 solutions, pick the best 3, refine that, pick the best 1
- After everyone has done this, agreed, likes it, the move on to wireframes, flows, etc
- Wireframes
- User stories & flows
- Product design != ux/ui design
- Sketching
- Never "launching" always shipping
- Align product vision and goals
- Tools
- Product design
- Ux/ui.
- File mgmt
- Dropbox
- Product mgmt
- Trello
- Google docs
- Used to use pivotal tracker but it didn't work for then
- Bug tracking
- Lighthouse
- To do list
- Behance action runner
- Asana
- Product design
- Culture principals
- PayPal mafia (graphic showing how all the early founders went on to start new successes)
- It's not about making ideas, it's about making ideas happen
- Creating great team
- Not about rock stars, about finding a team that meshes well.
- Over communicate
- Transparency
- Tell employees about vc talks, where the company is going, etc
- Create solutions not problems. Think about how to fix, mot just pointing out problems
- Ship it
- Done is better than perfect
- Startups within startups
- Set unrealistic deadlines
- Estimate what you can do, then aim for 1.5x that
- All new hires sign an agreement that they understand and agree with the culture
- Simplify and enhance focus
- Focus
- Finding the most important thing to work on is critical
- Startups don't starve they drown (in ideas)
- Focus means saying no to distractions, not saying yes to the one thing (Steve jobs, paraphrased)
- Questions
- How to get focused?
- What experiments do you want to run, what can get done in 2 weeks
- How many hats do ppl wear?
- 3-4. Helping creative process, (developing), analytics driven (a/b testing), teaching everyone SQL to look at DB, recruiting
- Does all the shipping cause bad bugs to go live?
- Baseline testing. Err on shipping first, fixing later
- They have had critical bugs but they fix quickly, they keep users happy
- What's the turnover?
- Haven't had anyone quit. Everyone who has left has been the company's decision
- Very transparent, strong culture
- What's the hiring process?
- Mostly referrals
- Don't use recruiters
- Inbounds/branding/team branding
- Etsy does a good job of this and is an inspiration
- Very thorough process, multiple phone interviews, homework, meet everyone, check references, keep it within 3 weeks
- Try to hire for where you want the team to be 12 months from now
- How many have you let go?
- About 5 in a year
- How do you find the balance of too much and too little process?
- No right answer.
- Play around with it
- Any special incentives to keep ppl engaged/motivated?
- Allow ppl to take as many classes
- Allow ppl to go in groups of up to 4 and spend company money to do anything they want
- Give lots of freedom and responsibility
- How to get focused?
CTO Artsy Daniel Doubrovkine: Pressing the big reset button (@dblockdotorg. Slides).
- How do we feel about resets?
- It's negative, happens as a consequence of failure
- Always happens with drama. 'slam the breaks'
- Compared to changing the wings on a flying airplane
- Examples
- Exhibit 1 Microsoft windows vista/longhorn
- Dev began in 2001
- Very feature-bloated. By 2004 clearly going nowhere
- Reset in 2004 on top of windows server 2003
- In 2006 it was released as vista (major failure)
- Exhibit 2 Microsoft office 98 & netdocs
- Netdocs was created by microsofties who thought the Internet would kill office
- Net docs 1 was java, effectively an office reset
- Net docs 2 reset in C++
- Net docs 3 reset as a service
- Spent $500M over the years, 500 devs
- Exhibit 1 Microsoft windows vista/longhorn
- Why do resets happen? Survey of about 2000 reset projects. Data from CTO club in NYC and personal network
- 4/10: sunk cost kept them from resetting
- 100% say due to some type of failure
- 7/10 faced strong opposition to resetting
- 2/10 blame pure product failure
- 3/10 blame pure tech failure
- 2/10 blame ppl failure
- 3/10 resets led to success
- 4/10 led to failure
- 100% of projects that weren't reset failed
- When is it right to reset?
- always, provided you are 100% sure the project is failing (Ed: tautology?)
- Find truth-Sayers, trusted ppl who will be honest & help you know if it is really failing
- Measure how much cost sunk & how much value it is providing
- Questioner points out: this is the sunk cost fallacy, better to focus on the value
- Don't be a chicken
- (chicken & pig committed v involved story)
- Only those that are committed should decide
- It can be a smart career move to make the hard choice
- Can be very dumb career move if you fail or the organization doesn't support you in the reset after you hit the button
- How to reset
- Act now, decisively
- Invest in trust capital
- Have to compromise in order to gain trust. For every year of trust capital you have a about a month to burn when in a reset
- Build it by helping others achieve their goals
- Double or nothing
- Split execute
- Under promise & over deliver
- Audience points out: this is the exact opposite of Malcolm Casey's strategy
- Aftermath of a reset
- People forget very quickly how bad it was
- Don't expe t to be glorified afterward. This is just part of your job
- Making decisions about when/where to continue dev paths is always part of your job
- Foster a culture of frequent but smaller resets
- Experiment often
- If things are posed as experiments, and you let devs try their ideas, you will keep evolving (Ed: and they will be happy)
- Stop at the top
- When you're at the top, don't get comfortable...try resetting then to make it even better
- Ed: wha?!
- When you're at the top, don't get comfortable...try resetting then to make it even better
- Questions
- How is the data calibrated?
- (audience grumbling/saying 'let him speak')
- Somewhat anecdotal
- How does that affect the code base?
- Depends, not necessarily software/code (e.g., could be purchased CRM software, when to throw in towel?)
- When ppl from the outside advocate reset, it's not good. Is that a personal opinion or from the data?
- How do you connect the concept of reset to tech debt?
- They are well connected. I don't like working in environments where you need to explicitly pull out tech debt. That should go without saying
- I pivot on market more than on technology. When you pivoted/reset the tech/rewrote did you get the founders to buy in too?
- Yes, that is critical, also critical to show that it is promising quickly
- How is the data calibrated?
Lean Startup Metrics class notes
I went to the Lean Startup Metrics class on Friday. My notes:
Why metrics important. "what gets measured gets done" Avoid biases in judgment. Numbers don't lie.
5 types: vanity, actionable, predictive, cohort, funnel
Vanity: impressive but don't really mean anything to your business. Vanity metrics are usually aggregate. Funding is too. It doesn't directly relate to success (bc funded startups fail all the time)
Actionable. When you measure it you know what to do. Measure the outcomes and see if your hypothesis is holding true. You have to agree ahead of time what metric to base success on. Cost per acquisition. Find that out early. Big data: FB learned that when women update their profile photo men spend more time on the site. And women are more likely to change theirs when they see other women change. You can use this to get men spending more time on site. Actionable metrics are per-person metrics.
Predictive metrics: Value: What we are doing adds value for customers. Net promoter score: "how likely are you to recommend this?" 1-10. 8-10 are promoters, 1-6 are detractors. Promoter - detractor / total. This is the nps. This is something that you can compare cross industries and products with.
Must-have metric: how disappointed would you be if you can't use this product? You want to see 40% of your users very disappointed. (determined by Sean Ellis) This helps you see if you have something unique.
FB growth metric: w/in 4 weeks of launching a campus, got 90% of the users FB value metric: missed it
Groupon: they started out as a WP blog selling 2-1 pizza coupons.
Groupon's growth model: paid. Their customer is the business, and they have salespeople to get more businesses to sign up.
Funnel metrics: They measure the life cycle of your user. AARRR acquisition activation retention referral revenue Measure how customers move through these stages Determine weaknesses.
Cohort metrics Segmenting user based on when they joined. This helps smooth out seasonal/temporary metrics. Kissmetrics. Mixpanel. Most startups will have to build custom solutions to capture their metrics, because each startup's important metrics are different.
Lean Cust Dev Class
Two biz purposes: mktg & innovation
Find customer for product before you build anything.
Lean startup is: pre-marketing
Startup def: institution designed to deliver a product under conditions of extreme uncertainty.
Startup is all about experimentation and uncertainty.
How do you know what the next step is?
Myth: lean == cheap Lean is about efficiency, investing just enough to validate assumptions.
Myth: lean is only for tech. Not so, it's just about experimentation.
Lean startup story: grocery delivering where they manually delivered groceries to their alpha customers by hand before building any tech to support, just to make sure that they really were delivering what customers wanted.
Why are founders reluctant to ask people about what they think about the idea? One reason: they are afraid of someone stealing the idea. Not so, ideas are cheap, execution is the hard part. Validate the assumption!
Myth: lean replaces vision with data and customer's wants. Vision is very important but your strategy (I.e. solution) needs to be responsive and validated.
UX how to ask questions do experiments to get useful feedback: First step: is there a problem worth solving? The riskiest part of the biz is rarely the solution. Don't say: 'here's a solution, do you like it?' They will say yes for the wrong reasons. How do you ask Q's that explore their experience? "here is a scenario. Have you experienced this? what did you do in this situation?" How did they actually solve the problem? Ask behavioral questions. Need to learn if there is a problem worth solving. The word problem is not always the perfect word (in the development of a game, for example).
Scientific method. Try to disprove hypotheses.
Startups that succeed are the ones that iterate enough before they run out of resources. Failure is not bad, it is part of the learning process. Learn from each failure and use that knowledge to pivot to a new iteration.
Surveys don't give a lot of info. Qualitative interviews give tons of info.
Caveat: in rapid iterations your objective is efficiency in finding product market fit. It's possible to have small enough selections of people that you don't fully validate your assumption (ie you get a false negative).
Good falsifiable hypothesis: [repeatable action] will produce [measurable outcome]
When you ask customers questions, you can't rely on what they say. When people say they will buy X, they will not. (they believe they or someone will, but people don't honestly know themselves that well and aren't that good at predicting their own future behavior). Cf Dan Gilbert quote
How do you decide the next step?
Entrepreneurs are too attached to their ideas. Until they give it a chance to see if it will fail, they can't know if it will succeed.
When you design experiment, start w the single most critical idea. What will break my biz if it is not true? What is most critical and most unknown?
"get out of the building" and work on validating your ideas.
We humans are really good at projecting and experimenting in our heads but bad at predicting others' behavior.
Customers don't care about your solution, they care about their problems. -- Dave McClure
Your obj is to find an early adopter, not a mainstream consumer.
The early adopter: a) has the prob you want to solve and b) they are aware of it. [And bonus if c) they already are trying to hack some solution to it]
You want to find them and build them an MVP.
When you have a lot of uncertainty, it doesn't take much more data to reduce it significantly.
Surveys assume you know the right questions to ask. -Ash Maurya
Myth: Customers don't know what they want. They *do* but they probably can't articulate it well. So don't ask them what they want, ask what they do (in situations like the problem you are trying to solve).
MVP are experiments meant to invalidate your riskiest assumption. Important not to focus on the word 'product'. It's not necessarily a product.
MVP phase 1: talk to people find out how they've solved this problem, see if it's really a problem. MVP 2: product pitch. Can be as simple as paper. Show it to someone and ask "would you use this?"
MVP 3: 'concierge' build something that is just barely enough to be usable, probably w lots of manual help Groupon is a good example. They just emailed ppl, printed coupons and sent them to them, to see if ppl would like this. Didn't build any tech. Tested it w PDFs and email. 'wizard of oz': build a frontend but no tech backend. Example early Charles Schwab wanted to see if ppl would do online trading so they built a frontend for ppl to enter orders but on the backend it was human traders seeing those orders and executing them.
Qs: How to find best ppl to do interviews? Come up w 'persona', find ppl who fits that persona. One type of pivot is 'customer-segment' pivot which is changing to a different customer persona and doing more interviews.
Vivek Sodera from RapLeaf @vsodera
- + -
recruiting at Rapleaf
- start out interviewing 25-40 engineers/month, goal was to increase that multiple times
- pattern recognition
- by the end: averaging interviewing 1200-1300 engineers per month
- philosophy: pass over someone good rather than pick someone bad
- passed candidates were picked up by google, facebook, etc
- questions
+ - how many were/are you trying to hire?
- wanted to keep lean, didn't want to drastically effect culture
- hire 1-2 per month
- hiring 1 out of 1000, or 1 out of 1500 people
+ - if you know you only want 1-2 people a month (for a total of, say 10-ish), why not sniper and just choose those people and go after them directly?
- part of the rejection marketing is to get the word out about the company
- it's not just hiring, it's also marketing for the company
- rejected candidates sometimes ended up becoming customers
+ - how many man-hours did you spend on this process?
hiring is the most important thing, so that's always their primary focus
not everyone in the company is involved in the recruiting...the first 15 people were, after about 20 hires not everyone was involved
depends on where the company is. When you're small you're going to work your ass off finding people and figuring your process
over time their engineers got much better at pass/failing test results
+ - when you were bringing in 1-2k people per month, how many were involved?
- the ops specialist who was managing it
- new engineers would get trained on how the process works and to bring in new people
+ - what about online contests/games for devs?
yeah, useful, but sort of low ROI
helpful if you have a platform you can make available for them to use/learn on
example: they had their own brilliant dev come up with a very difficult problem, and market it around (to ACM entrants, etc)
- 50% respond, try it out
- got a handful of decent apps
great to find research-minded thinker types...but if you're small maybe you need a *doer* more importantly
+ - q about interns: are they a good use of time?
- 40% of the interns they ended up hiring full-time
- vetted them well
- summer internship
+ - did you ever measure your cost-per-hire
- outsourced lead gen was the cheapest by far
+ - did you use adsense, facebook ads?
- yes...but fb ads didn't do well
- what ended up working was going to conferences (posting little ads above urinals in the bathrooms)
- played with auto-retweeting, auto @mentioning people who say certain things...
+ - what's the referral process?
at one point giving 10k if the person was hired and stayed for more than 90 days
toned it down to 5k (which was contested)
if you send us someone who passes the first round, you get $20, 2nd round you get $100, etc etc -- didn't get much success from that
+ - what was best?
- outsourced lead gen
- send very informal emails (lowercase words in subject line)
- a paragraph or two tops...the PS was the call to action
- A/B test up the wazoo
- + -
snipering vs shotgunning
- snipering
- passive candidates
- lots of experience
- looking for vp eng, cto level roles
- number of hires small (1-5)
- outreach via recruiter, referral, phone, email
- want to give the impression that it's hard to get into your company
- message is: we want high quality people
- shotgunning
- looking for active candidates
- 0-10 year experience range
- looking for lower-level positions (software engineer, manager, director)
- large number of target hires (3-100)
- the person who runs this is the operations specialist (they build the machine/funnel)
- need to clearly show what your culture is
- (preferred approach)
- snipering
- + -
rejection marketing
- want to give perception that your company is sought-after and exclusive
- bring as many people into the pipeline as possible
- and reject as many people
- rejectees will tell friends, friends will want to apply to see if they are "up to the challenge"
- increases the number of high-quality organic applicants
- feed the funnel with low-quality candidates (they will get rejected and tell their peers)
- + -
building a recruiting machine
- define and document core values
- + -
modify recruiting/interviewing flow for scalability
- phone interviews don't scale to this many candidates...hence online tests
- "here is what we would ask you in a phone interview, you have an hour to get back to us"
- hire ops specialist to run the machine
- they own this
- outsource menial tasks
- make the copy interesting
- no one gets excited by '3.5 years of X, 2 years of Y'
- think of it as a brochure for your company...
- fire recruiters, they suck
they are working with X companies, maybe you have a retainer
if they come across a great candidate, they will send their best people to the companies paying the highest commission (which will not be your company)
they all source from the same places, they all have the same candidates
caveat: there are some awesome recruiters (but are very rare0
doesn't like getting candidates from recruiters...they (the candidate) are often lazy...it's a lazy approach to looking for a job
- iterate, iterate
- recruiting channels
- outsourced lead generation
create a fake linkedin profile
add linkedin open networkers (they have tons of reach)
once your linkedin network is highly expanded, you can find more people
also: odesk...find cheap labor, pay people to go on linkedin and look for types of people, scrape the data...email thousands of those people, see who you can bring in
get those people onto a monthly newsletter so you can keep emailing them
- the newsletter was engineering-related
- direct mail blasts
- content for developers
- social media outreach
- ads/flyers
- university outreach
- distributed job posts
- developer-centric events
- events that had free beer/sandwiches, learning opps for developers
- research data sets
- outsourced lead generation
Kurt from Intent Media @kurt
- Kurt from Intent Media @kurt
- internet ecommerce advertising company
- 2.5 yrs old
- 42 people
- just raised $20MM
- questions
- + -
what to do if you come in to a startup as CTO and there's a team there that doesn't agree with you?
get credibility
look at every set of commits that comes in...comment on them if necessary
(audience suggestion: pair program w/ some of the developers to get familiarity with the code, and respect)
- + -
approx what team size do you stop being a coding cto?
it happened organically
it got to the point where there were so many meetings that he couldn't block out 3- or 4-hour blocks to code
have to make the decision whether you are coding all the time or you are going to protect your engineers from distractions so they can be coding all the time
"I never had a 4-hour block, so it was never worth getting started" on some coding
try to keep engineers out of meetings
- + -
how should the biz side relate to engineering? what level of communication is appropriate?
- they still all work in an open space
- priorities get decided upon by sales, bizdev, chief product officer
- along w/ that, he adds technical thigns like "we need to upgrade rails"
- present the biz folks with a list...this is what we can do...what do you want me to do
- + -
Q: You have a CTO above you...so what does he do and what do you do?
- Kurt handles inward-facing stuff
- CTO handles outward-facing stuff
- investor relations
- he's technical, not a technologist
- they have each other's backs
- advice on what to do with an underperforming engineering team and an overbearing demanding biz side?
used to work at thoughtworks (agile consulting), they would go into places and consult on agile
there's a 'consulting answer'
in a startup environment, the answer is fire those people
if someone is underperforming, there's no point in saying "let's wait a while and see"
- never gone back and said, "glad I waited to get rid of that person"
along with that, if you come in to an underperforming team and it continues to underperform, it looks bad on you
little more tricky on the biz side of things
- what are other ways you protect your team or other things the team needs protecting from?
- protect from: meeting/overhead that you can absorb
- don't let some sales guy go talk to them to get something developed that they need developed
- minimizing distractions
- letting engineering run the show (sort of)
- can you comment on your hiring process vs vivek?
part of it is a numbers game
need to get out there, meet people, find people
referrals is a huge part of the way to do it
open-source some of your company code that is not core to your business...people will use it, fork it, develop it...hire those people
- do you involve engineers in hiring?
- yes
- multiple interviews, multiple tech interviews, a non-tech interview
- have recruiters in-house
- + -
what to do if you come in to a startup as CTO and there's a team there that doesn't agree with you?
- one of the first employees -- in charge of creating a dev team
- half engineering/qa/data science
- + -
culture
- if you have an engineering culture it has to permeate the entire company
- can't just be siloed to the dev side
- they work on 2-week iterations, releasing 4x a week
- there's a company meeting every tuesday...very open
- the biz people explain how they are goign to close a deal this week
- then if it doesn't close that week they explain why the following week
- if you have an engineering culture it has to permeate the entire company
- + -
first steps
- make it a strong engineering culture
- the first 2,3,4 hires really set the tone. they are important
- first hire needs to be a tech guy, needs to be a doer
- + -
coming in to a new situation
- if you come into a place that is in a bad situation
- need to change engineering, force people to write tests, write good code
- all the consultants except for one fired themselves
- "this is an engineering company, engineering always has a say"
- can't let biz or sales guys run over the engineering
- have to constantly broadcast information
- whole company needs to know where the build is
- what's coming down the pipeline
- make it clear to see what the tradeoffs are
- if you are one of the first...at some point you are not an engineer anymore, you are overhead
your job is to make it so that the engineers can write code (unimpeded)
when the CTO is trying to write code, there's no air cover for the rest of the engineers, they are vulnerable to biz folks sneaking in requests, distractions, etc
- + -
hiring
- at an early stage, when you really need to hire, you can't always get the perfect person
- + -
you need to build to your team's strength
- e.g. he likes ruby, they built first stuff in ruby
- for the backend ad server, they could only find java guys to hire, so they built in java
- + -
need to keep people challenged
- don't let people get bored
- if they get bored, they will leave
- have to let engineers be engineers
- if you lose a couple of really good people, it is a waterfall from there on out
- anyone here (in this room) can walk out the door and find 10 new job offers for more money
- he brought in people he worked with before, they all knew they would work together well
- + -
development
allow engineers to code
every afternoon has time blocked out when no one is allowed to talk to engineers so that they can work
+ - be constantly refining how things are done
have an engineering/product retrospective every six weeks where we sit down and talk about what's wrong and how it can be made better
sometime that's "there are too many meetings"
sometimes it's "this bizdev person keeps bothering me with requests"
important thing is to have the input/feedback loop and keep building the culture toward what you want it to be
keep asking: does this feel right?
- you have to keep defending what you are there to do
CTO School Session 3: Peter Bell
I was lucky enough to make session 3 of CTO School tonight, where Peter Bell, CTO of Skinn.io, spoke about Project Management, Quality Assurance, Estimation, user stories, metrics-driven development, and more. As I've come to expect, it was a very informative and thought-provoking session. I'm looking forward to the next one. Here are my notes:
- + -
metrics-driven development
- be willing to pull out a feature when it isn't working
- book: continuous delivery
- + -
deployment
- continuous deployment
- automated unit/acceptance tests
- also biz metrics tests
- if orders/day decreases significantly, rollback
- perf tests
- after deploying, look to see if common user behviors are taking longer than normal. if so, rollback
- when do you do this perf/biz metric testing?
- when the pain of not doing it exceeds the pain of adding new features
- "probably not before you hit product market fit"
- continuous deployment
- + -
pivots & rewrites
- it's gonna happen
- need to think heavily about what tech to use...don't over-scale early
- neil ford: "if you add a feature early, you are incurring tech debt"
ex: pull in info about twitter friends/followers
perfect fit for async msg queue
however, start by doing it sync -- takes a few seconds, but no real pain point there until more load, so don't change it yet
"what is the simplest thing that will work"
if you put in a hacky solution, put enough tests around it to avoid breaking the whole system (and reduce the fear of refctoring)
- + -
user stories
1/2 the room is using user stories (~ 15-20 people)
As a 'role' I want 'feature' so that 'benefit'
if you can't come up with a reason for a user to use that feature, they are probably not going to use it (don't build it!)
INVEST
I-independent
N-negotiable (if you can't fit all the info on a 4x6 card, get a bigger sharpie and a smaller card (you are doing too much)
V-valuable
- if it doesn't have a benefit, don't do it.
- If you can't clearly articulate the benefit, don't do it
- two types of feature ideas
- one type you think of offhand and then forget about it
- the other type bugs you every time you use the system, everyone clamors for it
- those are the ones you should build
E-estimable
- needs to be clear that we can figure out how to do it
- if it's not estimable, maybe it's a spike
S-small
- break things down...good size of story is 'half a day'
T-testable
- + -
user story decomposition
- tracer bullets - not tasks
- decompose into something that a business user can test
- not necessarily something releasable
- ex: registration system
hardcore--lots of steps, health care, etc etc
start with the simplest thing (first/last/email/password) now I know who you are
so simple it won't even validate these things...just laying out a form with a message. (first story)
now it's something a biz user can test! "yes, I like that message" or "no, that message is wrong" etc
now going to require validation (next story)
automated test: make it ID-based (look for #registrationError). Don't make it check for a string...too brittle, that string may change
now add one more screen with some healthcare info (next story)
at each point ask, "is this an MVP," could we release here?
this way new stuff is available for internal biz users to view and test on a daily or near-daily basis
- 'happy path'
- add validation
- add data
- add robustness
- ex: algorithm:
- what's the simplest algo to write that can do something useful
- something that just returns a 0 or 1, maybe, rather than the full prediction value
- then what's the simplest improvement you could make?
- repeat
- what's the simplest algo to write that can do something useful
- + -
questions
institutional memory -- how do we make sure we keep the deep knowledge behind something available
- the problem is, most institutional memory has alzheimers...
- the info is on a wiki
- but the wiki gets out of date
- our institutional memory becomes misleading
what acceptance test do you recommend?
they use rspec
committed the pretty-print rspec html output and made it available for everyone in the company to see, so they know what's up
+ - when you are small, getting all stakeholders to sit down and agree on everything itself becomes an overhead...?
- yes, all process is an overhead
- ex: daily standup: what did I do yesterday, what am I doing today, what's blocking me?
- you can use an event to drive process
- let's get beers tonight and make a decision about the specs
can you give ideas for product roadmapping?
- i like eric ries' definition of a lean startup
- a human institution developing products or services under conditions of extreme uncertainty
- intuit even decided they were a lean startup
- a human institution developing products or services under conditions of extreme uncertainty
- how much uncertainty do you have?
if your customers are signing up for v17 of your software and they're expecting a roadmap and you're expecting them to stick around for a long time, maybe they need it
does a roadmap have to be a fixed direction, or can it be a set point? ( a vision)
assumption: no one is creating a blind startup
== a startup that makes a 2-year roadmap, follows it no matter what anyone says, and checks to see if they are relevant at the end of 2 years
what's the value of having a roadmap, knowing that you are in an uncertain world, knowing thigns will change?
diff b/w vision and roadmp.
- vision: "we are going to staten island"
- roadmap: "this is how we are going to get there"
reality: VCs are paying for our dev time...THEY want to know what we'll be building for next 3 months
comment: any planning past 6 months goes nowhere...we plan for 3 months (and can usually hit it), but plans for 6-months+ never happen. and these are guys that manage 500+person orgs with millions of dollars behind them. And they had predictable businesses with established pipelines
- + -
iterations
- 1/4 doing 2-week iterations
- 1-week iterations are awesome if you can keep the pace up
- the world is moving toward 2-week iterations
- month is really long these days
- kanban-based approach
- you have some kind of backlog (stuff to do "next")
- this comes out of manufacturing
- you focus on limiting work in progress
goal is to minimize cycle time
in manufacturing, thngs half-built are worth nothing (you don't get paid until you ship *completed* cars)
same with programming...all half-finished features add 0 value to biz...kanban says: let's get the NEXT feature done as fast as possible, and get it out
- you have some kind of backlog (stuff to do "next")
- iterations v kanban
- when an organization has little structure, iterations can be good
- because they force you to focus for a few weeks
- kanban can help you, too, because it makes you focus on just one thing
- books
- lean software development (from 2003)
- when an organization has little structure, iterations can be good
- 1/4 doing 2-week iterations
- + -
estimation
- why estimate at all?
- cost go/no-go (if this takes 3 weeks to build, we can't do it.
- market window
- if it takes 6 weeks but it's necessary for somethign happening in 3 weeks, don't build it)
- ROI comparison
- not just "because" -- use estimation if it helps you
- time, price or both?
- most of the time it matters how much or how long it will take
- sometimes it doesn't matter how much it costs (within reason)
- worth knowing which is most important (can spend more to dev fast, versus dev slow but save money)
- what's successful -- 5% over time budget but 1000s use it, or 5% early but no one uses it?
- if you need estimates granularly at wildly accurate numbers, something is not right
- ideal days:
- problem with using ideal days is not every day is ideal (meetings, installing software, etc etc)
- story points
1,2,5,epic
- split epic into smaller amounts
people always ask "how many hours is a point"? so use ideal days when you're just starting
to get over the hump, just use ideal days and drop the days. 3 ideal days = 3 points
don't allow 0-point cards
- because you can end up with 100 0-point tickets and everyone expects it to happen immediately
- roll up multiple small ones into a 1-point story
problem with using larger numbers (10,20,50) is people will end up saying "oh, 47.5"...no one can be that precise
- small story points numbers makes people not estimate so granularly
t-shirt sizes (small, medium or large)
- why estimate at all?
- + -
quality assurance
- automated tests
- unit
- integration
- acceptance
- build in quality
- you don't improve quality by doing more inspection
- best way to improve software quality is not to do more QA
- although you do still need that
- best way to improve software quality is not to do more QA
- two important things:
- build in quality by writing automated acceptance tests
- do qa before you write the code
- Test-driven development
- if you have a QA resource (tester, manager, whomever)
- bring them into the meeting when you spec the feature...ask them how they would break/test it
- write down what they say...that's one of your automated acceptance tests
- you don't improve quality by doing more inspection
- what to do if you get a legacy codebase with no test coverage or lousy coverage
very first thing: make sure the developers know *how* to write tests (get a consultant, figure it out, whatever you need to do)
every time you get a bug, WRITE A FAILING TEST
- you will start getting clusters of tests in the important parts of your app
every time you tweak a method, write a test or two
- the boyscout method...leave the code cleaner than when you got there
- it's ok to not have code coverage in certain places, too
you might have code you know you're going to throw away in a week
write tests for the parts that scare you, that you don't know well how they work or think they'll break
write tests for the critical parts of your codebase
- automated tests
- + -
pair programming
- having a pair is about 15% less efficient than two people programming separately
- but there are 20% fewer defects/bugs that way
- ends up being roughly a wash, efficiency-wise and much better code
