This article is a comprehensive overview of using deep learning based object detection methods for aerial imagery via drones.Did you know Drones and it’s associated functions are set to be a $50 billion industry by 2023? Currently drones being used in domains such as agriculture, construction, public safety and security to name a few and are rapidly being adopted by others. With deep-learning based computer vision now powering these drones, industry experts are now predicting unprecedented use in previously unimaginable or infeasible applications.We explore some of these applications along with challenges in automation of drone-based monitoring through deep learning.Finally, a case-study is presented for automating remote inspection of construction projects in Africa using Nanonets machine learning framework.Man has always been feed fascinated with the view of the world from the top — building watch-towers, high fortwalls, capturing the highest mountain peak. To capture a glimpse and share it with the world, people went to great lengths to defy gravity, enlisting the help of ladders, tall buildings, kites, balloons, planes, and rockets.Today, access to drones that can fly as high as 2kms is possible even for the general public. These drones have high resolution cameras attached to them that are capable of acquiring quality images which can be used for various kinds of analysis.With easier access to drones, we’re seeing a lot of interest and activity by photographers & hobbyists, who are using it to make creative projects such as capturing inequality in South Africa or breathtaking views of New York which might make Woody Allen proud.We explore some here:Energy : Inspection of solar farmsRoutine inspection and maintenance is a herculean task for solar farms. The traditional manual inspection method can only support the inspection frequency of once in three months. Because of the hostile environment, solar panels may have defects; broken solar panel units reduce the power output efficiency.Agriculture: Early plant disease detectionResearchers at Imperial College London is mounting multi-spectral cameras on drones that will use special filters to capture reflected light from selected regions of the electromagnetic spectrum. Stressed plants typically display a ‘spectral signature’ that distinguishes them from healthy plants.Public Safety: Shark detectionAnalysis of overhead view of a large mass of land/water can yield a vast amount of information in terms of security and public safety. One such example is spotting sharks in the water off the coast of Australia. Australia-based Westpac Group has developed a deep learning based object detection system to detect sharks in the water.There are various other applications to aerial images such as Civil Engineering (routine bridge inspections, power line surveillance and traffic surveying), Oil and Gas (on- & offshore inspection of oil and gas platforms, drilling rigs), Public Safety (motor vehicle accidents, nuclear accidents, structural fires, ship collisions, plane and train crashes) & Security (Traffic surveillance, Border surveillance,Coastal surveillance, Controlling hostile demonstrations and rioting).To comprehensively capture terrain & landscapes, the process of acquiring aerial images can be summarised in two steps.After image stitching, the generated map can be used for various kinds of analysis for the applications mentioned above.High-resolution aerial imagery is increasingly available at the global scale and contains an abundance of information about features of interest that could be correlated with maintenance, land development, disease control, defect localisation, surveillance, etc. Unfortunately, such data are highly unstructured and thus challenging to extract meaningful insights from at scale, even with intensive manual analysis.For eg, classification of urban land use is typically based on surveys performed by trained professionals. As such, this task is labor-intensive, infrequent, slow, and costly. As a result, such data are mostly available in developed countries and big cities that have the resources and the vision necessary to collect and curate it.Another motivation for automating the analysis of aerial imagery stems from the urgency of predicting changes in the region of interest. For eg, crowd counting and crowd behaviour is frequently done during large public gatherings such as concerts, football matches, protests, etc. Traditionally, a human is behind the analysis of images being streamed from a CCTV camera directly to the command centre. As you may imagine, there are several problems with this approach such as human latency or error in detecting an event and lack of sufficient views via standard-static CCTV cameras.Below are some of the commonly occurring challenges when using aerial imagery.There are several challenges to overcome when automating the analysis of drone imagery. Following lists a few of them with a prospective solution:Pragmatic Master, a South-African robotics-as-a-service collaborated with Nanonets for automation of remotely monitoring progress of a housing construction project in Africa.We aim to detect the following infrastructure to capture the construction progress of a house in it’s various stages : a foundation (start), wallplate (in-progress), roof (partially complete), apron (finishing touches) and geyser (ready-to-move in)Pragmatic Master chose Nanonets as it’s deep learning provider because of it’s easy-to-use web platform and plug&play APIs.The end-to-end process of using the Nanonets API is as simple as four steps.2. Labelling of images: Labelling images is probably the hardest and the most time-consuming step in any supervised machine learning pipeline, but at Nanonets we have this covered for you. We have in-house experts that have multiple years of working with aerial images. They will annotate your images with high precision and accuracy to aid better model training. For the Pragmatic Master use-case, we were labelling the following objects and their total count in all the images.3. Model training: At Nanonets we employ the principle of Transfer Learning while training on your images. This involves re-training a pre-trained model that has already been pre-trained with a large number of aerial images. This helps the model identify micro patterns such as edges, lines and contours easily on your images and focus on the more specific macro patterns such as houses, trees, humans, cars, etc. Transfer learning also gives a boost in term of training time as the model does not need to be trained for a large number of iterations to give a good performance.Our proprietary deep learning software smartly selects the best model along with optimising the hyper-parameters for your use-case. This involves searching through multiple models and through a hyperspace of parameters using advanced search algorithms.The hardest objects to detect are the smallest ones, due to their low resolution. Our model training strategy is optimised to detect very small objects such as Geysers and Aprons which have an area of a few pixels.Following are the mean average precision per class that we get, Roof: 95.1%Geyser: 88%Wallplate: 92%Apron: 81%Note: Adding more images can lead to an increase in the mean average precision. Our API also supports detecting multiple objects in the same image such as Roofs and Aprons in one image.4. Test & Integrate: Once the model is trained, you can either integrate Nanonet’s API directly into your system or we also provide a docker image with the trained model and inference code that you can use. Docker images can easily scale and provide a fault tolerant inference system.Customer trust is our top priority. We are committed towards providing you ownership and control over your content at all times. We provide two plans for using our service,For both the plans, we use highly sophisticated data privacy and security protocols in collaboration with Amazon Web Services, which is our cloud partner. Your dataset is anonymised and goes through minimal human intervention during the pre-processing and training process. All our human labellers have signed a non-disclosure agreement (NDA) to protect your data from going into wrong hands. As we believe in the philosophy of “Your data is yours!”, you can request us to delete your data from our servers at any stage.NanoNets is a web service that makes it easy to use Deep Learning. You can build a model with your own data to achieve high accuracy & use our APIs to integrate the same in your application.Pragmatic Master is a South African robotics as a service company that provides camera-mounted drones to acquire images of construction, farming and mining sites. These images are analysed to track progress, identify challenges, eliminate inefficiencies and provide an overall aerial view of the site.From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.Machine Learning EngineerNanoNets: Machine Learning API