The idea of an artificial intelligence (AI) has been around for quite some time. Both in fiction and no-fiction the idea of creating a form of non-human intelligence that could interact with humans has been romanticized. But the computation of having a machine learn on its own, after understanding a core of directed principles from reviewing examples is Machine Learning. And Machine Learning is really one of the most fundamental concepts inside of Artificial Intelligence.
Those examples, from which the machine learns are as simple as images from a camera or video, or as complex as language translation, to name a few simple examples. These examples are more commonly referred to as ‘observations’. The observations of the object or thing are accompanied by one or more labels which which describes the object/thing. The more observations/examples that are provided, in theory, the better the machine learning model. The the ‘Model’ is the perspective learned. Generally the ‘Model’ is very focused on the single perspective of learning and it usually takes are large amount of examples to learn the perspective with greatest accuracy.
The methods of machine learning most seem to concern themselves with are supervised and unsupervised learning. Differences in these algorithmic machine learning methods can get you loosely defined to very pedantic definitions depending on who you ask and under which purposeful intent is being sought. But generally the difference is creating a machine learning model which consistently needs supervision by an external source, such as a human, to provide a prediction on a new observations not previously observed. Or unsupervised, where the algorithm can infer from existing observations what a new observations might be in order to make a prediction.
Machine learning needs lots of data in order for its algorithms to perform effectively. Models can be overfit or underfit and generally do require on-going maintenance depending on the ultimate goal of creating a machine learning model. Optimizing a machine learning model once architected can be a near full-time job for one person depending on the intent of the model. This is one reason why AI/ML intiatives need proper infrastructure setup in order to provide the most optimimal results. Cloud architecture has helped to provide this leap in computational power and at-scale capability. And, great frameworks like pyTorch and TensorFlow are enabling some amazing models to be realized. Lastly, the importance of storing the data for the observations, the respective labels, etc. cannot be overlooked an most often a Data Lake, and a Data Integration component with emphasis of separation of data duties and repeatable data collections is so very important.
We’ll be discussing more on AI, ML, Data Lakes that support those initiatives and more in the upcoming months.
