Grammar-based procedural modelling of building façades from images

This work outlines a grammar-based solution for the rapid modelling of building façades from images. The project involved a user-annotated image of a façade being interpreted by the system, which in turn encodes the positional relationship between the façade’s elements into an internal representation, known as a split grammar [1]. The latter was evaluated further to construct a hierarchical subdivision, producing a meaningful 3D model upon traversal (see Figure 1).

The main objectives of this project include: an automatic system for deriving split rules from a single annotated image; a procedural approach for modelling a façade from a given ruleset and a procedural approach for generating a random façade structure from all the previously encountered instances. Practical applications include: architectural model creation, simulation creation and large-scale urban modelling with building variations, benefitting architects, game designers and film-set designers alike.

In the same way that the structure of a language is often defined through its grammar, a building façade could be encoded as a context-free grammar – G(T, NT, R, S₀) – consisting of terminal (T) and non-terminal (NT) symbols, production rules (R) and a starting symbol (S₀) respectively. The terminal and non-terminal symbols together define the set of symbols acceptable by the system. Symbols in T represent the individual façade elements (i.e., regions in which further splitting is not possible), while those in NT correspond to any compound grouping of terminal regions. Production rules are a vital component of any grammar, as they specify ways in which non-terminal regions could be converted into terminal symbols. The chosen system defines split and conversion rules to achieve its goal, the two key qualities which distinguish a split grammar [1].

The proposed system has been designed to cater for the following façade elements: doors, windows, balconies, balcony doors and shop items (e.g., banners, posters, signs and shop windows). The user marks these features using a different coloured rectangle for each separate group. To help the system distinguish between the different floors, the user would also be asked to select the floors to be modelled. This initial interaction is beneficial for both parties because, while a user has full control of the floor-selection process, the system would be made aware of the elements pertaining to each chosen floor.

A recursive splitting algorithm evaluates each floor region to determine the positional arrangement of all the façade elements falling within the current scope. This creates the production ruleset, which is the set of rules that uniquely define a façade structure. From these rules, an abstract syntax tree is constructed. This hierarchical data structure converts the scope-based positional relationship between elements in the same region to geometric information pertaining to the global scope. Finally, a depth-first traversal of this tree would lead to the construction of a 3D model.

The outcome of the project suggests that procedural modelling of façades from images is possible through the creation of deterministic split grammars, which uniquely encode single façades. On the other hand, the procedural modelling of a random building is achievable through the construction of a stochastic split grammar, which encodes all previously encountered variations [2].

References/Bibliography

[1] P. Wonka, M. Wimmer, F. Sillion and W. Ribarsky, “Instant architecture”, ACM Transactions on Graphics, vol. 22, no. 3, pp. 669-677, 2003.
[2] F. Wu, D. Yan, W. Dong, X. Zhang and P. Wonka, “Inverse Procedural Modeling of Facade Layouts”, ACM Transactions on Graphics, vol. 33, no. 4, pp. 1-10, 2014.