Sketch2Code is a solution that uses AI to transform a handwritten user interface design from a picture to a valid HTML markup code.
The process of transformation of a handwritten image to HTML this solution implements is detailed as follows:
- The user uploads an image through the website.
- A custom vision model predicts what HTML elements are present in the image and their location.
- A handwritten text recognition service reads the text inside the predicted elements.
- A layout algorithm uses the spatial information from all the bounding boxes of the predicted elements to generate a grid structure that accommodates all.
- An HTML generation engine uses all these pieces of information to generate an HTML markup code reflecting the result.
- <A href=”https://github.com/Microsoft/ailab/tree/master/Sketch2Code”>Sketch2Code Github</a>