Posted in All things Data, Personal Stories

What Humans can Learn from Machines


In my job, I work closely with all things Data. And the magical words you’d hear most likely after that is Artificial Intelligence and Machine Learning. We work with clients to help them use Machine Learning – gleaning insights from their data to gain a sustainable advantage in their business.

In other words, we help them discover what they do not know yet with the Data they already have. And that’s made possible with programming and building algorithms that can learn from the past to predict the future.  

The concept of teaching algorithms to learn from the past and replicating the future is a powerful one – and a lot of it has roots in observation of human behavior.

Look back at the history of how the various branches of Machine learning and AI evolved. Most of the thinking that contributed to this discipline was around making machines intelligent by mimicking inner workings of a human brain. Dig a little deeper and you would find branches like Neural networks & Reinforcement learning – entire paradigms of Machine learning inspired off human thinking processes.

After having worked in this industry quite a bit, and getting familiar with the inner workings of these algorithms, an insight that struck me was how much of the reverse is true.

Of course, many types of Algorithms have been taught to learn based on how Humans think and learn. However, there is a also lot that we Humans can learn from how algorithms get trained , tested and then and perform in the Real World.

Here are some examples:

You learn from what you Observe: Any machine learning algorithm you develop has this computationally intensive phase called the learning phase. You train the algorithm with a certain set of inputs and outputs – the machine picks up the patterns in the data to build a model of the world. Now, when you give it a new set of data to make a prediction – it generates an output based on the representation of the world that it has built.  Isn’t this how real life works?  Oftentimes we lament on our lack of ability to respond favorably to unexpected scenarios.  The reality is – you always learn from what you observe.

Generalizations from scant Data leads to Overfitting – Developing your life’s principles from scant data gives you an inaccurate representation of reality.  When models learn from too little data, then they fall into the peril of Overfitting. What that means is – they perform very well in test scenarios i.e. the environment where they have learnt, but fail miserably in the real world. In real life too, when you develop very strong viewpoints based on little data – it is quite certain that you might be wrong.  One observation of that in the workplace is how every person’s world view gets skewed by what they have seen in their previous roles and organizations – with learnings that might not be completely transferable. Hence, if you have a limited perspective and a new world before you – anticipate that you might be wrong. Look for new data that challenges your established beliefs, and that would help you be aware of the biases you have.

Exposing yourself to new Data enriches you to the next level: When a model does not give us good results – there are usually two ways of improving the accuracy. Either you feed the model new data – which is called ‘feature engineering’, or try a new way of looking at the data which is ‘Algorithm selection’.  Considering that you’ve done your homework right in the first place, in my experience – ‘Feature engineering’ (almost) always trumps ‘Algorithm selection’. The more relevant data you expose an algorithm to, the better it learns.  And the reason that happens is that more and varied data helps the algorithm develop an understanding of a wide variety of scenarios. In real life, the advice you hear is – get out of your comfort zone.  So, while the advice is to go ahead and do something that challenges you, what we are really saying is that expose yourself to a situation that you have not dealt with before.  More data helps you develop a worldview that is diverse and captures the intricacies that enable superior decision making. 

You need many models to map the complexity of the world: With one viewpoint, your understanding of reality is most likely biased. So, don’t depend too much on the opinions of those who are very similar to you. Research, ask questions – seek out diverse viewpoints. Pursue varied opinions because you achieve wisdom through a multiplicity of lenses. Otherwise, if all you know to use is a hammer – everything seems to look like a nail. Taking the parallel from machine learning, we observe that various models perform differently in different data dimensions, and a combination of models usually gives us superior results. So, the learning here is that if you want get a more accurate understanding of reality – think of multiple approaches for solving a problem. “Get a toolbox, not a hammer.”

The world is not Binary: One of my key instincts after years of management experience was to obsessively simplify messaging – get to the heart of the problem and find simple solutions. What I have realized over time is – the world is complex, and working with data and algorithms has helped me appreciate and embrace that complexity. For example, when we build machine learning models – say propensity to upgrade a product, there is usually no single data point that is overwhelmingly predictive of the outcome, but a combination of scores of signals or features that can accurately predict how a customer would behave. Similarly, machine learning also reveals that there can be hundreds of micro-segments in your data – customers with their own unique needs, wants and aspirations, which can be addressed uniquely. The world is not binary, even though we have strong instincts to view it so — ‘We are losing our jobs because immigrants are coming in and taking them’, ‘Equal pay for equal work will solve all women’s problems’. Binary answers are usually not accurate – and can sometimes be downright dangerous.

“Beware of simple ideas and simple solutions. History is full of visionaries who used simple utopian visions to justify terrible actions. Welcome complexity. Combine ideas. Compromise.”

In summary – as researchers and practitioners, we have built AI and Machine Learning systems by replicating the learning processes of human neurons and building patterns in the data that is fed to them. Unknowingly, we might have created a mirror image of real Life in these self-learning systems.  

One which powerful, dynamic and feeds not just from Human learnings, but also informs Humans on how to Learn!

Photo by Franki Chamaki on Unsplash

Posted in All things Data

A Letter of Recommendation – Algorithms


My fascination with Algorithms started when I was quite young. In my teens perhaps – when algorithms and the emerging world of computers seemed to be enticing and promising in equal measure.  My brush with them began in a high school computer class, when we were introduced to these archaic boxes of off-white bulky computers with a grey or black hard keyboards.

It was the early days, computers were a relatively new invention and being able to see one live in front of us was quite exciting. The first language we learnt was BASIC, and then graduated to more cognitively expensive ones like C and C++. You could make the computer do enchanting things, by giving it the most complex earth shattering instructions and then watch with pleasure as it bends over backwards to do your bidding. Indeed, how dramatic! 

And to add to that, these computers were primitive and heated up rather quickly so they needed enclosed and air-tight rooms with air conditioners in what was known as the “Computer department”. If you grew up in a small town with the harsh unforgiving Indian summer – spending time in there was quite a treat. Computer classes were the favorite even among students who didn’t fancy programming.

For myself – I must say that even though I was quite fascinated by the concept of programming, we never really hit it off.   I remember having read through the dense “Algorithms and Data Structures” book in my Engineering to capture any nuggets of wisdom that programming would bring. There was a promise , a connection to all the wonderful happenings in the Tech industry. Dramatic advances in technology  with a vision to transform the world. However, making loops in my head and if-then-else-break statements began to feel like a chore very soon.

Until, one day, almost ten years later – I discovered the magical world of Machine Learning. 

Machine learning, as a concept was a new paradigm where computers do not need to be programmed with explicit instructions about what needs to be done, but can be taught to learn purely by observation. And at the core of it is the concept of Learning. So first you train your algorithms to learn from the past, and with this knowledge of the past learnt primarily by observation, your algorithms can predict the future and take actions. All this may sound very mysterious but there is plain logic and and a lot of math behind all this.

My discovery of Machine Learning was not accidental. I started with reading books and spinning experiments of my own. And slowly, applying these experiments in many work projects exposed me to the inner workings of these digital beasts.  

And the more I knew – the more it astounded me. Imagine a machine that can observe how a system has performed in the past and develop a complete knowledge of the system from Day One. The power of this capability is mind boggling , and frightening in equal measure. 

Today, this is the core of what I do. And yet, the more I discover it, the more enamored I am by how much of this world fits into certain patterns, and how much of it can be discovered through pure math. It is also surprising to me how these algorithms reveal our hidden beliefs and desires, some of them which we might not be aware of ourselves. 

There are very many fears on what this means for our future, and where this technology will take us. And it also raises provocative questions.

  • What can we learn from these super powerful algorithms?
  • What are the benefits and limitations of using these technologies?
  • How do we leverage these technologies without succumbing to the inherent biases they come with ?
  • What are the key challenges we would face — as strategists, programmers, individuals, society and humankind in general?

There are many versions of answers to these. I am hoping to discover my own answers through these pages.

Photo by Kevin Ku on Unsplash

Posted in Personal Stories

Five Hacks to help you Read More


When life happens, there are a million distractions coming your way. Notifications on the phone, work goals, your child’s homework, that super critical mail which came in at the last minute when you were leaving office. There is always something urgent that is screaming for your attention.. And as expectations on how soon you should respond to messages and emails grow tremendously, there is this time crunch squeezing our lives into tinier and tinier moments. One of the side effects of that has been the inability to see the bigger picture,  relish a quiet moment, or even read long books. There was a time when I couldn’t break away from “the arc of unbroken concentration” which a good book could bring.

Today,  I feel perpetually lost in this trough of shallowness.

So how do you break out of this ? How do you dim this digital hum and  develop “the willpower to focus on a sustained argument”.

Here are some of hacks to read more:

Make goals: Goals help us stay consistent, and give a constant nudge to move on to the next step. So go ahead and make goals.Long term ones in terms of books per year, or even short ones like minutes read per day. Even if you have only 30 minutes you can spare in a day – every second of it is worth being spent on reading!

Borrow rather than Buy: Counterintuitive as it may seem, abundance is one of the biggest killers of motivation. “Ah, I bought this book and it is right there on my shelf. Why spend my time trying to focus and read it when I binge watch mindlessly on Netflix?”.  As our attention spans dwindle, it is easy to discard book-worthy moments for easier options.  There is a sense of scarcity that exists with borrowed books, a deadline by which they will expire and go away.  And that can help you read more.

Begin reading new books on a Thursday or Friday: This hack words for me, because if you have a busy week , then starting to read something while you are easing into a weekend is a good way to stay motivated and focused. Weekends are great for long reading stretches, if you have the willingness to be alone.  

Disconnect from technology: When reading a book on my mobile or tablet, every tiny notification costs you minutes of lost focus. So go for low tech – the most basic Kindle or hard copy book. Even on phones, Apple has this screen time feature that shuts off my phone at 10 pm every night, and those are the most peaceful moments of my day –  when I can pick up and read a book.

Share your insights and learnings: When you’ve read a book that you liked or enjoyed, talk about it to friends, your spouse, or even write down how you felt. Sharing what you’ve learnt will help you internalize it, and also motivate you towards finding the next book. 

Well.. Quite honestly, finding the next book always makes me anxious . When I’ve finished one, and am wondering which one is going to be my next – there is an expectant longing in air.

From this vast rich expanse, which one would I  choose ?

Curiously – “Some like to believe it’s the book that chooses the person.” !

Photo by Patrick Tomasso on Unsplash

Posted in Personal Stories

Learnings on the Highway


The black car zoomed through the darkness, interspersed with the glare of many others moving in perfect symphony across the dark highway. And I am driving inside it within my own perfect bubble. The music blares on the car audio, with a personally curated Spotify list algorithmically doling out songs for my pleasure. I am acutely focused, my mind in sync with the vibes on the road. The gray asphalt silhouettes against a dark blue and orange horizon. Night has fallen long ago, with rebellious shards of light appearing  to replace a sun long gone. I gaze at the dashboard and the traffic ahead, unconsciously looking out for the impatient lane changers who zip across lanes as they find an opening, trying to achieve a hypothetical advantage over others.

The car noiselessly glides ahead like it has a life of its own, the needle on the speedometer at a constant seventy. The entire world shrinks down to a tiny pinpoint of light ahead of me. 

In the dark confines of this shared space , there is a strange kinship that develops with all the motorists going my way. It is almost like we are some randomly distributed molecules who have been drawn by a strong magnetic force to move together in alignment towards a common destination. 

And in that moment, there is an epiphany in my mind. In today’s fast moving world, we blame so many of our problems and stresses on speed and pace at which things need to move. This is something that impacts our family, our work and friendships.  The general advice we give ourselves is to slow down. 

What I learned today from the highway is: 

Moving fast is not a problem – misalignment is. 

The highway is a perfect example of harmony – where everyone is moving at incredibly high speeds but yet there is perfect order because there are rules of the game which are followed by all travelers on the road.. Pass from the left.. Make way for newcomers entering on the right.. Stay on the left lane if you want to go faster than the rest..These elementary rules can make what seems like an impossibly dangerous commute become so simple and seamless. 

So next time you are harried with too many things moving too fast in your life – think of the Highway Analogy. Align your priorities or take the Exit.

What would you do to make Life go smooth !?

Photo by Jonas Von Werne on Unsplash

 

 

Posted in Fiction Experiments

The Pink Flats


It’s a third time today that I am opening that tempting tab in my browser to peek at this object of desire that has caught my fancy.

I find myself staring at this pair of sinfully elegant, unapologetically feminine pink flats.

Then my eyes hover over the hefty price tag – surely my overstuffed closet doesn’t need a $450 Gucci shoe. But the mind is wandering and I can imagine this pair going perfectly with that still untouched magenta skirt which I had never been confident enough to pull off.

A meeting reminder pops up on my screen and breaks me of my reverie. “Ah”, I scold myself – “The PG&E meeting is in 15 minutes and you are lusting over a pair of shoes that you don’t need. Wake up!”.

To give some background, I work at BigRock – an investment management firm.  And yes – today is just another ordinary day. I have an upcoming meeting with a client and there is this lingering sense of anxiety to bat this well.

“Many eyes are on this deal, and if you nail it – the Sales Director role is yours. Just make sure they get the messages right .. and don’t make it seem too much of a hard –sell .. and hopefully, they won’t dwell too much on the terms in Section #4… ”

A constant drone of to-do lists is hammering at the back of my head.., imagining all the ways this meeting can go wrong. And every time there is a blip in this train of thoughts – I am seeking refuge in these pair of Parisian pink flats.

“You need these shoes to be successful!!”  – a chirpy voice screams inside my head.  Click .. click ..click. it would be so easy to buy.

“I’m sorry, but..”, another calm voice patiently explains. ” Remember how you had to give away those Coach boots which were in perfect shape because you realized you had two pairs which looked exactly like the same?”

“But well, you know..” – the chirpy voice pops up again – “Don’t be so hard on yourself,  you’ve been working so hard and that contract that you won last month. You deserve this!”

“So.. On Page#5, don’t forget to mention that we analyze the attractiveness of an asset not only based on its risk, but also on the return potential the asset may bring to the portfolio”  – That one is my BigRock employed thread, urgently entreating to be heard among the other two voices.

“Blah, blah, blah”.. the chirpy voice interrupts again. “Pleated Magenta skirt with a sleeveless white blouse and pink flats with a floppy bow –  now that’s a killer outfit!”…

“Nooo… Too many shoes in your closet, remember?!”. The calm voice seems all flustered and is screaming now.

“.. and the long-term performance goal for this portfolio is to achieve at least 80% of the return of the global stock market” – weakly mutters the BigRock thread again.

The calendar invite has popped up. It is time to join the PG&E call now.  I take a deep breath and flick these voices away as I begin dialing the number…

Hello Everyone, this is Anjali from BigRock Investments”, I brightly announce myself on the phone.

Like I said..  it is just another ordinary day today.

Posted in Personal Stories

5 things I discovered about myself today


through_the_chaos_of_life_by_helale-d5svr6x

I’m certain Google is turning my brain into mush. Its shaping the way I think – putting ideas in my head and making me imagine that all of them were my own. So, every time I have to process a question that will take more than a few microseconds of thinking time, I’m tempted to turn to Google for answers. Now that’s an evil genius!

My laptop is now an extended part of my self.  Long periods of absence without using it make me anxious, and I see myself increasingly shoving mine in the back seat even on short trips. And yet ironically, we have a deep unfaithful relationship – wouldn’t mind leaving my current love if a better one comes by.

Solving problems helps me thrive. When I’m frustrated , I go solve a problem – clean the dishes, go fix the cracked dresser. That makes me feel better, every-time!

I have lost my capability to monotask.  Doing just one thing at one time makes me paranoid, with a voice screaming in my head that I’m missing something urgent. Even driving alone without the radio on makes me uncomfortable  – I’d rather not be left to my own thoughts. I’m certainly doomed. Sigh.

Sunday nights always make me think about what I’m doing with my life. And trigger random unrelated discoveries and existential questions.  Which are also entirely meaningless.

Thank you if you’ve read them so far though.  Have a rocking week ahead! 🙂
Posted in Personal Stories

Idea #5: Fail Early, Fail Fast


A few weeks ago, Apple released its Maps App with the new iOS 6 upgrade. Within hours, the internet was awash with scathing criticism of Apple Maps. It was branded a dismal failure, the L’Enfant terrible of iOS 6 upgrade portfolio. FB news feeds flooded with lamentations about awful maps after upgrading to iOS 6.

Tech bloggers lashed out at Apple, calling it a company which had become arrogant in its greatness. Doomsday predictors began asking – is this the end of Apple’s commitment to perfection and its meteoric rise?

All this brouhaha really irked me. For all those going berserk over the Map App fiasco, wouldn’t it be nicer if we could cut Apple some slack and appreciate the benefits of failing early??

I am not an Apple fanboy, but personally I felt the new Maps app was awesome. It brought in radical features like vector-based maps and breathtaking flyover 3D views (my first reaction on seeing the App was “Wow!”). Yeah, the awesome 3D feature resulted in the most terrible goof ups too. But then, online Maps are complex and you cannot perfect it unless you have millions of people sharing it and reporting back to you. Does anyone remember Google Maps when it was first launched?

The point is, with technology cycles moving so quickly, could Apple have afforded to keep the critical Map app under the hood till the next iPhone release? Absolutely not.

The fact is – if anything can go wrong, it will. And the sooner that happens, the better it is for you and your product. And the most valuable company of our times has sent the message – the future is about failing early, failing fast. And if you do make mistakes, admit them quickly and move on.

Well.. Google does it all the time! (remember Google Buzz, Froogle and Google Answers?)

Getting a minimum viable product out into the world is the best way to kick start your idea, whether it is a new book, a business venture, or a project at work. It helps you overcome the fear of failure, test your assumptions and identify constraints in your solution. So go ahead with your new creation. Get it read, experienced and used.

Because usage is like oxygen for ideas. “You can never fully anticipate how an audience is going to react to something you’ve created until it’s out there. That means every moment you’re working on something without it being in the public it’s actually dying, deprived of the oxygen of the real world.”

So fail early. Learn fast!.

And if you are an unflappable cool dude who hates to admit your mistakes or let the world know about your failures – Just fail early..   And then destroy all evidence that you tried!.. 🙂