26 March 2017

IT Skills Shortage

Many news sources state that there is a skills shortage in various sectors of IT. However, in close inspection this really is not the case at all. In fact, there is huge talent pool out there of candidates eager and willing to work. In most cases, for every job there are hundreds of applicants. Organizations are just looking for cultural fit. In other cases, one might have the skills but just not be someone that the employer gets along with. That does not mean that there is a skills shortage. In fact, it just means the organization is discriminating on the things they are looking for which are not even relevant to the role and the skills. In fact, outsourcing is a key factor in buying in talent on the cheap leaving local talent behind. Many human resource evaluations do state that the larger degree of the interview is about the person rather than the quantifiable skills that they can bring to the team. It is more about the likability which is a form of discrimination again people want to work with people that are more like them. This definitely does not imply that there is a skills shortage in the market. There are plenty of women out there that are very good in Big Data and Machine Learning but does one see them on their team. Employers are basically utilizing a wand of cultural nonsense and overlooking what really matters is the ability of the candidate to perform based on their skills as well as to innovate. Large organizations like Twitter, Facebook, IBM, Google, Microsoft, Amazon, and others harp on about skills shortage but they not focused on diversity and inclusion within their teams. They also buy in cheap outsourcing contracts from places like India ignoring the local thriving talent pool within the prospective regions. We need to recruit more women and generally people of all types and backgrounds in IT, supporting the local communities of where multinationals are based, and that starts from each individual that is well positioned towards evaluating applications on merit and objectivity. If people hire people, and people are generally not always objective, then perhaps, artificial intelligence needs to diminish the responsibilities of the human resources function in organizations.

25 March 2017

Future of Software Engineering

In world of Big Data we essentially have four separate encompassing roles: Big Data Engineer, Data Scientist, Data Architect, and Data Analyst. In most companies the differentiation of the roles generally has a distinctive overlap. However most of these role types are a summation of one main role as it were and that is of a Computer Scientist or more simply put of a Software Engineer. A Big Data Engineer ordinarily is a Software Engineer who is able to look at the big picture. This implies essentially that a Data Scientist, Data Analyst, and a Data Architect are really subsets of the Big Data Engineer. In future, we will witness the convergence of these roles into one where many of the separate role responsibilities will merge into the Software Engineer role. As memory requirements grow to meet Big Data it will mean that Software Engineers take on a broader scope of work. The many disparities in roles converging implies that complexity is more managed and accessible across a standard engineering team. Obtaining a Data Scientist with a Phd is no longer going to be the conservative forte for organizations. Many Software Engineers are able to do the work of Data Scientist as well as Big Data Engineers and manage to look at the big picture from the business standpoint. In almost all cases, algorithms can be taught, approaches can be taught, skills can be re-learnt, but really what teams require is the flexible mindset to adapt to change. Data Scientists have several limitations for organizations: they tend to have questionable programming skills at R and/or Python, familiar with specifically a set of statistical and/or machine learning approaches, and generally apply imperfect/overfitted models to a very small subset of data constraints which are not always adaptable to change in the realistic Big Data requirements of a business. They also have a hacking mindset and a very strong dependency on Big Data Engineers to provide the backbone for ingestion of data sources, refactoring, feature engineering, data pipelining, model scaling, and to ease their model building process. That is almost three quarters of the Big Data work. Ordinarily, Software Engineers can handle multitasking across the entire stack as well as practically apply data science concepts. The future means that the roles of Data Analyst, Data Scientist, Data Architect, Big Data Engineers will no longer be necessary as part of recruitment and they will eventually be eclipsed by the standard Software Engineer. It is not so difficult to attend conferences, read journal papers, conduct research, and build models on data. One only really needs the right business mindset or domain knowledge. Learning and applying new approaches is a continuous process of Software Engineers to up their skills in moving with the times. Organizations will demand more out of hybrid Software Engineers in being able to adapt for changes which are often guided by the business need and the data landscape. Applied Artificial Intelligence will imply that even the building of generalizable models for data is going to become a simpler process of engineering without requiring specialist skills.

Speech Recognition Toolkits

Kaldi
Moses
SRILM
Sphinx
HTK
Simon
Julius
Tensorflow (WaveNet)

Baconjs

Bacon.js

Crunch & Scrunch

Apache Crunch

Cybersecurity with Spot

Apache Spot

Swarm Bandit Robotics

Border control is an issue for most land locked countries including ones that have direct access to water ways. Being able to control every aspect of a border is not humanly possible in most cases. But AI can in fact help cover a larger distance with greater amount of force and attributed control. Artificial Intelligence in form of swarm intelligence and reinforcement learning can create an effective force for border security and control. Building a great wall is pointless and overly expensive. Eventually the wall comes down. But, an army of drones and robots, if well engineered, becomes a force to reckon with as well as adaptable to patterns of attack and infiltration. The ultimate goal being an autonomous army of swarm robots that can apply tactical understanding and manoeuvrability across the entire map of the country, advanced in strategic alliance and combat, when necessary, to protect the sovereignty of a nation and its people.

Military Swarming
Kilobots
pentagon drone swarm autonomous war machines