December 9, 2020
Data and models, two sides of the same coin.
Data is the lifeblood of the deep algorithms of Artificial Intelligence (AI) and Machine Learning (ML), but the truth is that only a few big players have the most power over this resource, leaving other companies at a disadvantage. In this scenario, synthetic data is a great tool to minimize the gap when little data is available and allows organizations of all levels and sizes to be part of the development of these algorithms. In other words, democratize machine learning.
Synthetic data: a brief definition
Is artificially created data rather than generated by real-world events, that resemble real data. It can help reduce time, costs, and risks associated with data collection, while also enabling results similar to those obtained with real data. It can also contribute to maintaining the confidentiality of the information since the data created can be anonymized.
Synthetic data is very helpful when you have unbalanced data (classes with unequal sample sizes). By making use of it, the sample size of the minority class can be increased (oversampling) quickly, since hundreds of new instances can be generated in a short time. An example of unbalanced data is those from anomalies, which are usually the minority group compared to their counterpart. Here you can find an applied example.
Another scenario in which synthetic data can be useful is when it’s required to maintain the confidentiality of the information. In this case, the real data can be completely replaced by synthetic data (built based on the real data) that keeps similar distributions and correlations to the original one. In this way, reliable results can be achieved without the risk of losing confidentiality. As an example, we can think of bank-related and personal data.
Synthetic data can also be used when data is scarce, highly heterogeneous, or difficult to obtain. There are several fields that have already begun to work with synthetic data, among them we can find: IoT, medicine, and autonomous vehicles, just to name a few.
Flip is an open-source library that allows you to generate synthetic images in a few steps from a small set of images made up of backgrounds and objects (images that would be on the background). To do this, it allows the user to apply transformations on objects such as flipping about an axis, resizing, or rotating. With the above defined, a composition is made with a set of options to establish the final state of the background and the location of the objects. Finally, the user can generate jpg, json, csv and pascal voc files already labeled.
Now let’s see how it works in the practice. In this link you can see an example of how to use the library to create new synthetic images for each background, you can also determine the number of objects to add.
For this example the following images were used (which you can find in the example repository):
Object: penguin in png format with transparent background.
Background: landscape in jpg format.
Some lines of code are explained below.
♦ Flip: allows you to rotate the object or the background about the x, y axis or random.
In images 1 and 2 a flip was applied to the background and in 3 and 4 it was applied to the object. In 1 and 3 it was with respect to the x axis, while in 2 and 4 it was with respect to the y axis.
♦ RandomResize: change the size of the object with respect to itself or the background. The changes can be asymmetric or symmetric with respect to width or height, with the possibility of defining maximum and minimum values.
♦ Rotate: rotate the objects at angles of 90º, 180º or randomly (between 0º and 360º). The last one allows determining the minimum and maximum values to make the rotations.
In image 1 the random setup was used, in image 2 theupsidedown or 180º setup , and in image 3 the 90º setup.
After defining the transformations of the objects, a composition is made between them and the backgrounds (to which it’s also possible to apply transformations such as Flip). The way in which the objects will be placed on the backgrounds can be in two ways: by percentage or completely random. The user can determine the minimum and maximum values (for both width and height) where the objects could be positioned.
Finally, the user can save the generated images in jpg format.
You can also download the generated labels adding the following transformers to the compose object.
The first line will generate the bounding boxes around the targeted object and the second one will save them in a JSON format, you can also save the labels in csv and pascal voc formats. The .json file contains an array of dictionaries (one per image generated) with two keys: name and pos.
name: contains the name of the image.
pos: contains the position of the objects in the background. The values x and y determine the top-left coordinates and the values w and h are the width and height of the bounding box (all units are in pixels).
For this example, all images have the same name as it was used only one image.
The csv format will look as follows.
pinguino.png 200 439 112 202
pinguino.png 126 449 112 245
pinguino.png 345 436 112 234
pinguino.png 621 384 112 239
Pascal voc format will generate a .xml file with additional information, the name of the image, its position, dimensions (width, height, and channels), the name of the images used as objects, and the values of the top-left coordinates (xmin, ymin) and bottom left (xmax, ymax) of each bounding box. You can see an example below.
To visualize the generated bounding boxes, you have to generate and save the JSON format first and then you can use the following notebook: show_labels.ipynb (you can find it in the examples of FLIP library). Bounding boxes will look like this: