Releasing Software Developer Superpowers

Article is aimed at anyone looking to gain the edge in their software development team creation or advancement in the digital age. Concepts can be applied outside of sw dev at some level. Open to discussion – views are my own.

UX is not just for Customers

User Experience is an ever growing component of product development, with creating user centric design paradigms to ensure that personalisation and consumer/market fit is achieved. From a development team view, leveraging some of the user experience concepts in how they work can achieve operational efficiency, to accelerate product development. For example, how is the experience for each of the developer personnas in your team? How do their days translate to user stories? Can interviewing the development community lead to creating better features for your development culture?

Build Products not Technology

Super important. Sometimes with developers, there is an over emphasis on the importance of building features, a lot of the time for features sake. By keeping the lens on the value or “job to be done” for the customer in the delivery of a product at all times can ensure you are building what is truly needed by your customer. To do this, select and leverage a series of metrics to measure value for that product, along with keeping your product developent in series, and tightly coupled to your customer experience development.

Leverage PaaS to deliver SaaS

This sounds catching but its becoming the norm. 5 years ago, it took a developer a week of development time to do what you can do in Amazon Web Services or Azure now in minutes. This has led to a paradigm shift, where you being to look at the various platforms and tools that are available to enable the developers to deliver great products to customers. Of course, there will always be custom development apps, but you can help your developers by getting them the right toolkit. There is no point reinventing the wheel when OTS open source components are sitting there, right? Products like Docker and Spring and concepts like DevOps are bringing huge value to organisations, enabling the delivery of software or microservices at enhanced speed. Also, the balance between buying OTS and building custom is a careful decision at product and strategic levels.

“The role of a developer is evolving to one like a top chef, where all the ingredients and tools are available, its just getting the recipe right to deliver beautiful products to your customer.”

Create Lean Ninjas!

Evolving the cultural mindset of developers and the organisation toward agile development is super important. Having critical mass of development resources, plus defined agile processes to deliver business success can really reshape how your organisation into one where value creation in a rapid manner can take place. However, its important to perform ethnographical studies on the organisation to assess the culture. This can help decide on which agile frameworks and practices (kanban, scrum, xp etc) can work best to evolve the development life cycle.

Implement the 10% rule

Could be slightly controversial, and can be hard to do. Developers should aim to spend 10% of their time looking at the new. The new technologies, development practices, company direction, conferences, training. Otherwise you will have a siloed mis-skilled pool of superheros with their powers bottled.

However, with lean ninjas and effective agile company wide processes, resources and time can be closely aligned to exact projects and avoid injecting randomness into the development lifecycle. Developers need time to immerse and focus. If you cant do that for them, or continously distract them with mistimed requests – they will leave. If you can enable them 10% is achievable.

Risk Awareness

We are seeing an evolution in threats to enterprise all over the world, and in a software driven and defined world, getting developers to have security inherent design practices prior to products hitting the market can help protect companies. Moons ago, everything sat on prem. The demands of consumers mean a myriad of cloud deployed services are adding to a complex technology footprint globally. If they know the risk landscape metrics from where they deploy, they can act accordingly. Naturally, lining them up with business leaders on compliance and security can also help on the educational pathway.

Business and Technology Convergence

We are beginning to see not only evolution in development practices – we are also seeing a new type of convergance (brought about by lean agile and other methods) where business roles and technology roles are converging. We are beginning to see business analysts and UX people directly positioned into development teams to represent the customer and change the mindset. We are seeing technology roles being positioned directly into business services teams like HR and finance. This is impacting culture, wherby the saviness in both directions needs to be embraced and developed.

Growth Mindset

We mentioned mindset a lot in the article. That because its hugely important. Having the right culture and mindset can make all the difference in team success. As Carol Dweck talks about in her book “Mindset”, you can broadly categorise them into two – growth and fixed. This can be applied in all walks of life, but for team building it can be critical.

In a fixed mindset students believe their basic abilities, their intelligence, their talents, are just fixed traits. They have a certain amount and that’s that, and then their goal becomes to look smart all the time and never look dumb. In a growth mindset students understand that their talents and abilities can be developed through effort, good teaching and persistence. They don’t necessarily think everyone’s the same or anyone can be Einstein, but they believe everyone can get smarter if they work at it.

Creating a team where being on a growth curve and failures are seen as learning can really enable a brilliant culture. As Michaelangelo said “I am still learning”. Especially as we evolve to six generations of developers. How do we ensure we are creating and mentoring the next set of leaders from interns through to experienced people?

Check a Ted talk from Carol here – link.

And most importantly … HAVE FUN!

Augmented and Virtual Reality: Now more about improving User Experience

We cannot eat popcorn wearing a virtual reality (VR) headset – Zaid Mahomedy : ImmersiveAuthority.com

IIn 1995, the cringe worthy Johnny Mnemonic was released where he used a VR headset and gesture monitoring gloves to control the “future internet”. Even though this movie was over 20 years ago, it is only in the past few years we are seeing commercially ready Virtual Reality (VR) and Augmented Reality (AR) technologies hit the market.

If you watch this clip, you will hopefully notice two aspects. The first is that the technology is clunky. The second is that the predicted user experience (UX) he has is rich (for the decade of movie production): information is available at speed, the gloves are accurate, and the path and usability is seamless. When he moves his hands, the VR3 responds instantaneous. It assists him at every turn. Yet twenty years later, we have not reached this quite yet. Why? Because the focus needs to shift from technology to other aspects to enable this industry to flourish.

1: Technology Moves Aside for User Experience.

A large amount of technology companies efforts in this space in the past two years has been mostly focused at determining can they squeeze enough compute power onto a pair of glasses. Other questions to be answered were around if the battery will last a decent amount of time and will the heat emissions be low enough not to inconvenience the user. Whilst there are still optimizations to be performed, the core of the technology has at least been proven, along with some clever innovations around leveraging smart phones to save on hardware investments.

In the coming years we will see a larger amount of these companies focusing on user experience we have with these technologies – ensuring the interfaces,gesture and motion recognition are close to perfect are high on companies to-do lists. The hardware road-map will ensure they are lighter, more robust and frankly – sexier to wear. Before we discuss other aspects of how improved UX will be the focus of the coming years, its not to stay that technology wont help on this. For example, the evolution of flexible compute paradigms specifically in the nano technology area will assist in building compute into glasses, instead of adding compute retrospectively.

2: Difference in Psychologies

Apart from the technology of VR and AR being quite different under the hood, the psychology of how they are used is also. With AR, we are injecting a digital layer between us and the physical world. With VR, we are immersing ourselves into a digital world. These are very different experiences and the user experience design must have this difference at its core. We must ensure that the layer we design for AR takes characteristics from both our physical environment and our own personas. With VR, its much more emphatic to ensure the person feels comfortable and safe in that world.

3: Interfaces to VR/AR UX

The UX Design of AR and VR technologies and applications will require careful management of multiple input styles. Using wearables, voice recognition, AR and AI, we will start to see seamless blending and integration with how technology interacts with us across various senses. Touch devices are still being used, but they will move aside for voice recognition, movement tracking and even brain waves to be used to control these smart devices. The communication will be much faster and intimate, and will force designers to completely rethink how we interact with these devices.

4: The Role of AI in UX

The UX of these devices will also require more human like interactions, to built trust between the devices and the users in an organic manner. We are seeing this with voice control technology like Siri and Google Home, but they are understanding our voice, with some sample responses. Soon they will learn to evolve their speech.

Artificial intelligence will take hold of the user experience to analyze the reaction to different experiences and then make changes in real time to those assessments. UX will become a much more intuitive and personalized experience in the coming years.

5: Convergence of VR and AR Standards

Already we are seeing a myriad of startups evolving in the space, some focusing on content development in software, some on the actual hardware itself. Some are brave enough to have both on offer. We also have the larger companies creating divisions to provide offerings in this space. Choice is great, but when it becomes painful trying on your fourteenth pair of glasses at your average conference, it is not. When one takes time to observe how companies are beginning to partner up to offer solutions ( a trend extremely common in the IoT industry) it is a small step towards some form of standardization. Excessive choice can be bad from a UX perspective, as with such segregation in initial design makes it harder for app designers to get it right on the hardware.

6: Realistic Market Sensing

At some point, we have to get away from the “Toys” feel for these devices. We put them on for ten minutes in an airport or at an event to get a wow from it. Whilst the applications in the gaming industries are there to be seen, companies are beginning to focus on where else the market will be. Certain devices have flopped in the past two years, and you would wonder why with such strong brands. The first reason was awful UX. The second was the market just was not ready, with a distinct lack of content to make them anyway useful. Just because a few of these devices fail, doesn’t mean the movement stops. (Below info-graphic source is washington.edu)

Consumer and Industrial applications have very different requirements from a market perspective, with content choice and look and feel very important for consumer markets, system performance and governance sitting higher in industrial use cases. With the costs associated with adding these technologies to industrial environments under the microscope, companies must focus strongly on measuring and building the return on investment (ROI) models.

7: Protecting the User and the Experience

With these technologies predicted to get even closer than headsets (smart contact lenses for example -link here), its quite important the UX designers can intrinsically build in comfort and safety into any application. Too many times we have seen people fall through something whilst wearing a headset (more so with VR technologies). And that’s just the physical safety. When the threshold between physical and augmented worlds gets closer and closer (mixed reality), we want to avoid a scenario of interface overkill.

Whilst the past few years may indicate that these technologies are fads, the reality is far from it. They will become part of our social fabric as a new form of mobile technology. Ensuring the users experience with these technologies will be the critical enabler in their success and adoption rate.

Designing for AR and VR entails there be better understanding of a user’s need when it comes to context of use. It’s about building connections between the physical and digital world, requiring an interdisciplinary effort of service design, interaction design and industrial design.

Machine Learning 1.0 over Coffee

Article aimed at anyone (technical or non technical) who wants to understand the steps in Machine Learning at a high level. Readable in five minutes over coffee. I think.

What is Machine Learning?

Today we live in a world of seemingly infinitesimal connected devices, in both personal and commercial environments. The currency associated with these devices is data, which whizzes around in near real time, is stored locally and in cloud environments. The types of data vary greatly, with text, audio, video and numerical data just a sample of the data modalities generated.

As this data is a currency, there is value associated with it, but how do we extract this value? A high growth area is called data science which is used to extract value and insight from this data. It has numerous ingredients in the recipe, with data mining, data optimization, statistics and machine learning key to generating any successful flavor. And like an good recipe, you need a good chef. These chefs in data terms are called Data Scientists, who use a wide variety of tools to glean insight from the data to deliver impact for your business. The data-sets themselves can either be uni-variate (single variable or feature), or multivariate (multiple variables or features). A persons age would be an example of uni-variate, whereas multivariate would expand a person’s feature set to include age, weight and waist size for example.

Why do it?

Machine learning (ML) is born out of the perspective that instead of telling computers how to perform every task, perhaps we can teach them to learn themselves. Examples include predicting the sale price of your house based on a set of features (sq. feet, number of bedrooms, area), to try to determine if an image is of a dog rather than a cat to determining the sentiment of a set of restaurant reviews to be positive or negative. There are a host of applications across many industries, some of these are shown below (source Forbes)

Before the magic is induced from the algorithms, perhaps the most important step in any machine learning problem is the upfront data transformation and mining, towards optimization. Optimization is required as most of the algorithms that “learn” are sensitive to what they receive as an input, and can greatly impact the accuracy of the model that you build. It can also ensure you have a thorough understanding of your data-set and the challenge you are trying to solve. Some of the data transformation and mining techniques include record linkage, feature derivation, outlier detection, missing value management and vector representation. All this is sometimes called “Exploratory Data Analysis”.

Techniques once Optimized

Once data is presented in the right manner, there are a number of machine learning techniques one can apply. They are broken in supervised and unsupervised techniques, with supervised learning taking an input data set to train your model on, and with unsupervised no data-sets are provided. Unsupervised techniques include learning vector quantification and clustering. Supervised techniques include nearest neighbors and decision trees. Another techniques is Reinforcement learning, and this type of algorithm allows software agents and/or machines to automatically determine the ideal behavior within a specific context, to maximize its performance.

Verifying your model is also an important step, and we often use confusion matrices to do that. This involves building a table of four results – true positives, true negatives, false positive and false negatives. A set of test data is applied to the classifier and the result are analysed to assess performance. Sometimes, the result of the model is still questionable. When this happens, machine learning has an answer in the form of ensemble methods, which essentially you build a series of models that you build your final prediction from. Examples here include boosting and bagging on the training data. Bagging splits the training data into multiple input sets, boosting works by building a series of increasingly complex models.

There are complimentary techniques used in any successful machine learning problem – these include data management and visualization, and software languages such as python and java have a variety of libraries that can be used for your projects.

Going further

Taking a step further from machine learning, you are into a complimentary area called artificial intelligence (AI), which leans more on methods such as neural networks and natural language processing which look to mimic the operation of the human brain. This is showing how human centric design in technology is evolving, and how much excitement there is for how humans and technology will work together in the future. It can be said this excitement is born from revealing that as we evolve our understanding of what it means to be human, it outweighs anything that technology alone can deliver. People have always been at the core of innovation, and this has led to an evolution in how improved our lives are.

Closing off Web Summit 2015 – Day 2/3

And so it ends. The Web Summit on Irish shores finished on Thursday (for now), and I must admit there was an athmosphere of “what if” and that of sombreness. But we cannot allow this to affect our perspective and thinking of the impact this conference has had on Ireland tech landscape.. Paddy Cosgrave has built a conference which he began with 400 attendees to now 35,000. Let’s put that into context in regards to people’s perspective of Ireland as a Tech Hub. With over 100 countries represented, and technology itself ensuring their own tech landscape is quite small, the voices of the 35,000 will translate into millions. And I am certain the conversation will be about Web Summit, the friendliness of the services and the vibrant Night Summit, and not the number/cost of hotels, government and traffic.

Wednesday was a great day and one of the best I have had at a web summit event. It’s started on the Machine Stage, where a panel including Nell Watson from Singularity spoke on how machines and humans will coexist and complement each other in our smart future. I liked how Nell spoke about how the seamless integration of machines, and the governance of same will be a key piece of the puzzle.

**Nell Watson from Singularity speaks on Machine Stage**

Next up on Machine was another panel, which included Dr. Joe Salvo from GE and Dr. Said Tabet from EMC. The panel was expertly hosted by Ed Walsh who is the director of technology vision for EMC. Whilst interviewing the guys, Ed brought out not only the technology vision required for IoT, but also the collaboration that can be enabled by consortiums like the IIC, of which Dr Salvo and Dr Tabet have been so instrumental in building.

**Ed Walsh hosting a panel session on the industrial internet**

As already mentioned, the proliferation of Virtual reality was evident, and I got a demo of Amazons audible technology! It was quite neat!

Friday was a more relaxed day, with numbers down a little but this allowed for a different kind of networking experience. It was a day to chat with as many startups as possible, and to catch some great talks. One that stood out was on centre stage, where a panel (including Christine Herron from Intel Capital, Albert Wenger from Union Square Ventures, Mood Rowghani from KPCB ) was hosted by Charlie Wells of the Wall Street Journal. Topic discussed – tomorrow’s tech landscape. A growth or just a bubble?

Well it looks like what Nell mentioned above is already happening from who I bumped into!

And so, we are off to Lisbon. Whilst I believe that there will be challenges there also, it is Cosgraves personality that will shine through. An excellent CI labs data science company spawned out of the web summit, and whist there is that data science feel to a lot of the web summit, it’s this personality of Cosgrave and his team that still makes this event stand above many.

10 Perspectives on “All Things Data”

Switching focus back to a series of technical blog posts, over the next 5/6 blog posts (there may be some Web Summit updates intertwined!) I aim to demystify “all things data”, to include reporting – analytics – data science – business intelligence, key difference and dependencies between these terms, explore an introduction to where machine learning fits into your data model in your company. Governance, security and data management will also be covered.

To begin, a short post with 10 perspectives that will get you thinking. (hopefully!)

1: Big Data is just a tool.

2: Analytics is utilized by Data Science and Business Intelligence

3: Data is never clean. You will spend more of your time cleaning and preparing data (up to 90%) than anything else.

4: 90% of tasks do not require deep machine learning

5: More data beats a cleverer algorithm

6: Data Science + Decision Science + Analytics = Business Impact

7: You should embrace the Bayesian approach

8: Academia and Business are two different worlds – know this.

9: Presentation/Visualisation is key (know your audience)

10: There is no fully automated Data Science. You need to get your hands dirty.

The 2020 Digital Employee: 10 Characteristics

Whilst this blog is focused at the characteristics that millennials (and others) functioning in the technology sector will need to have in the coming years, it could be associated to other industries also. Some of these features are already in play, however they are not seen currently as a full set. Happy reading.

1: Collaborator

Internal company perspectives will no longer be enough. With the lines between various technologies and industries blurring, to truly understand the necessary trends to keep up, both the person and the company will require one to forge strong active links with external companies, startups and universities. The tools required to achieve this will also evolve, with social media collaboration tools to come into the mainstream, along with the continued influence of smart devices and wearables for managing our workload.

2: Applies Relational Technology

A positive from active collaboration is the potential to see other technology methods across various industries. The wide lens approach will provide plenty of food for thought when the technologist looks to solve their immediate challenges. A good example of this is applying classical file compression algorithms to bioinformatic problems in genome sequence analysis for disease susceptibility patterns. There will be huge advances on the adage “Think outside the box”, where people will build algorithms to find best fit algorithms to solve a related challenge. Seriously.

3: Brand.me

Personal brand is going to continue to grow in influence for future technologists. There are a few aspects to brand to consider. First, your internal brand within your immediate company – how your colleagues view you, how you ensure you remain visible in the right areas within your company. Next, your external brand is how you are viewed in immediate applicable technology areas, both geographically and in parallel companies. Lastly, the social brand of a technologist will require a suitable online social media strategy, to compliment the first two, and ensure that you are visible in areas that may very well blur into yours in the coming years.

4: People Person/ Personality

For years, technologists had an interesting reputation! Most people believed them to sit in dark rooms, writing code, and building circuits, with “geek” and “nerd” aimed in their general direction. Not so now. Its now “cool” to be in technology, given that the technology we work on is impacting everyone’s lives. We can see it, feel it, touch it. Its real. And thus, the impression that technologists can make in various circles has increased. We are now in boardrooms (see my previous blog on trends), becoming online influencers, some are even getting celebrity status (Elon Musk). Also, there is going to be a continued evolution in the number of generations that we will have to work with, which will mean more youthful employees will have to lead the aging generations.

5: Employee Skills as a Service

OK this may sound controversial. But think about it. As the lines between companies blur, with collaboration having a magnetic effect in pulling them exceptionally close to one another, it is predicted here that employees may begin to work in different organisations, with companies contracting their core employees into other partner companies that may need a particular skill set for a fixed period of time. It is predicted here that it may go a step further, with employees interviewing companies, rather than the other way round. The shift in power will happen, and employees will maintain their time bandwidth per week/year, and will give their services on a consultancy basis to multiple companies. Also there is a trend that the “one company employee” is a thing of the past, with employees more free to move quicker between jobs.

6: Self Managing

The next generation of technologists will have independence in their DNA, and will possess the required soft skills to be able to self manage their time and tasks. Point 5 above will demand this, but this is not to say that upper management will not be required. What is being said is that the hierarchical org charts will be a thing of the past, with flat structures work best in evolving technology companies.

7: Mobility

The walls of companies will be well and truly knocked, with advances in technology ensuring that “work from anywhere” is a distinct reality. Augment reality will play a part in this, when renderings of colleagues will solve the lack of contact/visibility challenge that currently exists. Enabled by technology, an entirely new work environment is on the horizon. According to Wakefield research, 91 percent of C-level executives and IT decision-makers believe that today’s teenagers will be working in roles that do not exist today. 72% agree that the traditional office as we know it will become obsolete within four years. Think about it. How are the generation in school now communicating? There were born into technology.

8: Educational Diversity

With online education companies such as Coursera becoming hugely disruptive in the education sector, it is predicted that the classical – Degree – Post Grad – Work (with training) model will change greatly. Numerous people have been quoted as saying “I don’t use a huge amount of my primary degree”. This will mean that certain individuals (think of the 16 year old kid who became a millionaire) will be hired quicker by companies, and then incrementally receive their education throughout their company. This is quite common in Japanese companies, with kids being given apprenticeships at 16, and mix college with work over the next 6 years. Now if the employment laws would catch up! Whilst incremental training is happening now within companies, colleges/companies don’t recognize it as a sum of the parts.

9: Startup(s) as a Hobby

Currently, having external commitments in technology areas, such as startup involvement is seen as a bad thing by most companies. There are trends to suggest that companies are actively opening the door to employees who use their spare time to engage in other opportunities. And rightly so. The skill set that can be gained from contributing in different company and academic structures are incredibly valuable, and there is the added bonus for the company in that they have a viewpoint into more early stage alpha and beta companies.

10: Mass Parallel Processors of Information

Yep. Its happening. And we don’t even know it. The way education is being delivered these days demands huge levels of multitasking. The ability to respond to several different stimuli at the same time is called continuous partial attention. We used to teach in a way that demanded a tremendous amount of memorization, but now it’s more about cognitive agility and multi-tasking. The part of the brain, called the hippo-campus, that’s involved in memory is a little different than the multitasking part at the front of the brain.

We see it currently in technology. To every Splunk there is a Hunk. Hadoop was barely alive when Spark came along. Java now has over 50 different varieties. Argh! Do we need to be expert at all of them? No, but we need to be able to switch between them seamlessly. Or at least know what gets used where to meet the challenge we are working on.

5 Technology trends to consider right now

As we are a month inside the second half of 2015, I thought it would be a good time to look at some of the technology trends that are in motion, and will have more of an influence as we enter 2016.

1: SMAC becomes SMACT

Social Mobile, Analytics and Cloud (SMAC) has existed for a number of years in enterprise applications. Internet of Things (IoT) has accelerated as an enabler in technology, and hence will begin to be added to SMAC to create SMACT. I introduced this concept in one of my first posts here. And they need each other to succeed and/or progress. As more and more devices in IoT come online, SMAC demand will increase. And IoT will add value to SMAC, as it will spurn new technology directions that can utilize SMAC. The A in SMAC will be affect more than others, as with new data sets being generated, open data sets available for data multi tenancy will drive new requirements for on demand insights in real time.

2: Co-Creation:

A key tenant of open innovation (which was mentioned in a previous blog here) is co-creation. As companies take a more outside in approach to discovering next business direction, co-creation will be a huge part of this. Whilst its slowly increasing in chatter, co-creation will be key enabler in the coming years. Industry partners, vendors and consumers will create ecosystems that will drive new business models by utilizing analytics, and understanding customers at heightened levels. We have seen how disruptive NetFlix, Uber and Bitcoin have been in the past few years, and it is expected co-creation will also drive further disruption, but in different directions at increased velocity. Ikea’s home tour is a good example of them listening to their consumers to understand the requirements for why they were doing up their homes.

3: Technology and Business Strategy Leadership positions collide

It is expected that there will be a blurring of the lines between technology and classical business positions in companies, and this will result in a series of new positions to drive next generation technology direction. We are seeing that technology and business executives need to be proficient in both areas, and understand the dependencies of the decisions made in either will be crucial. The rise in roles such as Chief Data Architect (CDA), Chief Digital Officer (CDO) and Chief Governance Officer (CGO) has meant that board rooms have increased percentage of technology executives. It is predicted that an organisations Chief Technology Officer (CTO) will create a series of direct reports in the areas of data intelligence, data monetisation, futurism and collaboration strategy. These roles will be necessary to assist the CTO in managing digital disruption.

4: Data Monetization

This is a hot topic right now, and one of the pioneers that is driving a lot of new research in this area is Steve Todd from EMC, along with Dr. Jim Short of the San Diego Supercomputer Center. Whilst you can read extensively on this topic at Steve’s blog, Ill outline some of the considerations that are prominent for your business. The first is the idea of monetisation of your current and future data assets. Data is the new oil, a form of currency that can be used to drive business metamorphosis, but also can be something that is of use to others. So then it becomes a sale-able asset. We have seen first hand where major companies are looking to acquire companies not only for their technology, but their data also (example). Imagine if your store had a considerable data set, I expect major retailers such as amazon would be interested in buying that data-set from you, to understand street shopper trends. Another aspect to consider is valuing data at all stages of your companies cycle from inception, through beta to its growth cycle. An accurate snapshot of your data assets can increase the valuation of your organisation, and is especially useful in acquisition. From an internal company data perspective, a key pillar of your data monetization strategy is the architecture on which your data resides, as numerous data silos across your organisation are generally very difficult to even analyse for valuation. The concept of a business data lake can bring huge advantage here.

5: Search will involve more than Google

Currently, a large proportion of search involves online search for information that resides on servers. However, with the increased influence of IoT and the connected world, it is expected that more that the cloud will indeed be searchable. The billions of edge devices should enter the fray, if the data and security policies continue to be challenged into being more open. Connected cars, homes and mobile devices could widen the net for any search queries. We are seeing the emergence of alpha startups indicating this trend, such as thingful and shodan, which act as search engines for the internet of things.

Numenta and MemComputing: Perfect AI Synergy

Let’s look at two forces of attraction that are happening in the technology space, specifically looking at creating true artificial intelligent systems, utilizing advances in both software and hardware technologies.

For years, even decades we have chased it. AI has been at the top of any list of research interest groups, and while there have been some advances, the pertinent challenge has been that advances in hardware electronics in the 70’s and 80’s occurred, software design was lagging behind. Then, software advanced incredibly in the past decade. So now, in July 2015, we reach a key point of intersection of two “brain based technologies”, which could be built together in a way that may lead to “true AI”.

At no other point in history have we had both hardware and software technologies that can “learn” like we can, whose design is based on how our mind functions.

Numenta

First, let’s look at Numenta. Apart from having the pleasure of reading Jeff Hawkins excellent book “On Intelligence”, I have started to look at all the open source AI algorithms ( github here) that they provide. In a journey that start nine years ago, when Jeff Hawkins and Donna Dubinsky started Numenta, the plan was to create software that was modeled on the way our human brain processes information. Whilst its been a long journey, the California based startup have made accelerated progress lately.

Hawkins, the creator of the original Palm Pilot, is the brain expert and co-author of the 2004 book “On Intelligence.” Dubinsky and Hawkins met during their time building Handspring, they pulled together again in 2005 with researcher Dileep George to start Numenta. The company is dedicated to reproducing the processing power of the human brain, and it shipped its first product, Grok, earlier this year to detect odd patterns in information technology systems. Those anomalies may signal a problem in a computer server, and detecting the problems early could save time, money or both. (Think power efficiency in servers)

You might think, hmm, that’s not anything great for a first application of algorithms based on the mind, but its what we actually started doing as neanderthals. Pattern recognition. First it was objects, then it was patterns of events. And so on. Numenta is built on Hawkins theory of Hierarchical Temporal Memory (HTM), about how the brain has layers of memory that store data in time sequences, which explains why we easily remember the words and music of a song. (Try this in your head. Try start a song in the middle.. Or the alphabet.. It takes a second longer to start it). HTM became the formulation for Numenta’s code base, called Cortical Learning Algorithm (CLA), which in turn forms the basis of applications such as Grok.

Still with me? Great. So that’s the software designed and built on the layers of the cortex of our brains. Now lets look at the hardware side.

Memcomputing

After reading this article on Scientific American recently, and at the same time as reading Hawkins book, I really began to see how these two technologies could meet somewhere, silicon up, algorithms down.

A new computer prototype called a “memcomputer” works by mimicking the human brain, and could one day perform notoriously complex tasks like breaking codes, scientists say. These new, brain-inspired computing devices also could help neuroscientists better understand the workings of the human brain, researchers say.

In a conventional microchip, the processor, which executes computations, and the memory, which stores data, are separate entities. This constant transfer of data between the processor and the memory consumes energy and time, thus limiting the performance of standard computers.

In contrast, Massimiliano Di Ventra, a theoretical physicist at the University of California, San Diego, and his colleagues are building “memcomputers,” made up of “memprocessors,” that can actually store and process data. This setup mimics the neurons that make up the human brain, with each neuron serving as both the processor and the memory.

I wont go into specifics of the building blocks of how they are designed, but its based on three basic components of electronics – capacitors, resistors and inductors, or more aptly called memcapacitors, memresistors and meminductors. The paper describing this is here.

Di Ventra and his associates have built a prototype that are built from standard microelectronics. The scientists investigated a class of problems known as NP-complete. With this type of problem, a person may be able to quickly confirm whether any given solution may or may not work but can’t quickly find the best solution. One example of such a conundrum is the “traveling salesman problem,” in which someone is given a list of cities and asked to find the shortest route from a city that visits every other city exactly once and returns to the starting city. Finding the best solution is a brute force exercise.

The memprocessors in a memcomputer can work together to find every possible solution to such problems. If we work with this paradigm shift in computation, those problems that are notoriously difficult to solve with current computers can be solved more efficiently with memcomputers,” Di Ventra said. In addition, memcomputers could tackle problems that scientists are exploring with quantum computers, such as code breaking.

Imagine running software that is designed based on our minds, on hardware that is designed on our minds. Yikes!

In a future blog, I will discuss what this means in the context of the internet of things.

Distributed Analytics in IoT – Why Positioning is Key

The current global focus on the “Internet of Things (IoT)” have highlighted extreme importance of sensor-based intelligent and ubiquitous systems contributing to improving and introducing increased efficiency into our lives. There is a natural challenge in this, as the load on our networks and cloud infrastructures from a data perspective continues to increase. Velocity, variety and volume are attributes to consider when designed your IoT solution, and then it is necessary to design where and where the execution of analytical algorithms on the data sets should be placed.

Apart from classical data centers, there is a huge potential in looking at the various compute sources across the IoT landscape. We live in a world where compute is at every juncture, from us to our mobile phones, our sensor devices and gateways to our cars. Leveraging this normally idle compute is important in meeting the data analytics requirements in IoT. Future research will attempt to consider these challenges. There are three main classical architecture principles that can be applied to analytics. 1: Centralized 2: Decentralized and 3: Distributed.

The first, centralized is the most known and understood today. Pretty simple concept. Centralized compute across clusters of physical nodes is the landing zone (ingestion) for data coming from multiple locations. Data is thus in one place for analytics. By contrast, a decentralized architecture utilizes multiple big distributed clusters are hierarchically located in a tree like architecture. Consider the analogy where the leaves are close to the sources, can compute the data earlier or distribute the data more efficiently to perform the analysis. This can have some form of grouping applied to it, for example – per geographical location or some form of hierarchy setup to distribute the jobs.

Lastly, in a distributed architecture, which is the most suitable for devices in IoT, the compute is everywhere. Generally speaking, the further from centralized, the size of the compute decreases, right down to the silicon on the devices themselves. Therefore, it should be possible to push analytics tasks closer to the device. In that way, these analytics jobs can act as a sort of data filter and decision maker, to determine whether quick insight can be got from smaller data-sets at the edge or beyond, and whether or not to push the data to the cloud or discard. Naturally with this type of architecture, there are more constraints and requirements for effective network management, security and monitoring of not only the devices, but the traffic itself. It makes more sense to bring the computation power to the data, rather than the data to a centralized processing location.

There is a direct relationship between the smartness of the devices and the selection and effectiveness of these three outlined architectures. As our silicon gets smarter and more powerful and efficient, this will mean that more and more compute will become available, which should result in the less strain on the cloud. As we distribute the compute, it should mean more resilience in our solutions, as there is no single point of failure.

In summary, the “Intelligent Infrastructures” now form the crux of the IoT paradigm. This means that there will be more choice for IoT practitioners to determine where they place their analytics jobs to ensure they are best utilizing the compute that is available, and ensuring they control the latency for faster response, to meet the real time requirements for the business metamorphosis that is ongoing.

Nell, Google and a Half Pipe! EnterConf Belfast – Day 2

Quote of the day. “Counterfeiting is an insidious problem in life sciences, our network tenant cloud can help stop it” – Shabbir Dahod – TraceLink, Inc

As EnterConf entered its second day, I continually saw the benefit of having more detailed discussions with people in the Enterprise sector. Even during the night events (the speaker dinner in the Harbour Commissioners Office, great venue, followed by a few sociables in the Dirty Onion Bar), I kept monitored the dynamics taking place. The networking normally began with two people, but the circles were growing, joining to form what I like to call “RoundStandUps”. These were normally not short conversations, and collaboration was inherent in the voices and chatter. There also was a deep and satisfying undertone, which was an energy to keep “building great” in Ireland.

Check out the Half Pipe! Hope its at Web Summit! 🙂

Kicking us off on Centre Stage was none other than the inspirational futurist Nell Watson from Singularity University, who is also the CEO of Poikos, the smartphone 3D body measurement company. She talked about virtual employees, how we will replicate the human mind through AI in 20 years (and run business through AI). I liked how Nell bridged the machine and human inter-dependencies. It was an insightful talk, and having spent the past year looking at machine intelligence (from both a hardware and software implementation perspective), I am seeing more and more futurists thinking like this.

Nell Watson, CEO of Poikos on Centre Stage

A few talks focused on our evolving workplace. David Hale, from Gigwalk spoke on the Insight stage on “Deploying Technology to Power Mobile Field Teams and Maximise Work Efficiency”. David spoke on how mobile tools for consumer brands and retailers are being used to more effectively manage field teams, gather in-store data and direct resources to improve retail execution ROI. David also spoke about how our employees are changing, and how companies have to empower the “Millennial Employee”, whose requirements include flexibility, and having a social and online mindset.

David Hale, from Gigwalk on the Insight Stage

Shabbir Dahod – TraceLink, Inc, spoke on the Centre stage, his topic – “Delivering the Internet of Things (IoT) to the Enterprise”, and it was one of the highlight talks of the summit I found. Shabbir spoke about how Tracelink were the world’s largest track and trace network for connecting the Life Sciences supply chain and eliminating counterfeit drugs from the global marketplace, by using their Life Sciences Cloud, configured in a network tenant architecture.

Thomas Davies, Head of Enterprise for Google drew a huge level of engagement from the crowd with his talk on the next stage of collaboration. Thomas mentioned the evolution of how we collaborate, but even since the early 1980’s the structures were quite rigid and have not changed that much up to a few years ago. But now, customer and employee expectations have changed. They are fast, 24/7, global and personalised. He discussed how employees and organisations are more efficient when they collaborate. “We shape our tools, and then our tools shape us” – Marshall McLuan.

Thomas Davies (Google) in exhuberant form on Center Stage

One last talk Ill cover is a topic that is somewhat under the covers of Enterprise IT, and I am glad that Engin Akyol of Distil Networks talked on “Dark Cloud: Cloud Providers as a Platform for Bot Attacks”. Engin first spoke about good bots, which do serve a purpose for major cloud providers. But this talk was focusing on bad bots, which slow down application performance and skew analytics. As the volume of cloud platforms continues to scale, this leads to ease in setting up bot networks which can pilfer content from websites, or launch other malicious attacks.

So, ill sign off from EnterConf 2015, and onto Web Summit in November, with many events, collaborations and new experiences in between. As a two day conference, perhaps I built less contacts than I expected to. But the ones I did are more meaningful contacts, and EnterConf allows their attendees an environment to do that. I also sat in on round-tables on big data and security, which gave yet another dynamic. It really is a conference experience I will be returning to. Special mention to all the organisers, volunteers and the inspiring venue. Goodbye Belfast, hello Dublin!

Oh, I almost forgot, I really hope Krem Coffee are at Web Summit, awesome coffee!