Various experimental settings that involve drying solutions or suspensions of nanoparticles-often called nanofluids-have recently been used to produce structured nanoparticle layers. In addition to the formation of polygonal networks and spinodal-like patterns, the occurrence of branched structures has been reported. After reviewing the experimental results we use a modified version of the Monte Carlo model first introduced by Rabani [Nature 426, 271 (2003)] to study structure formation in evaporating films of nanoparticle solutions for the case that all structuring is driven by the interplay of evaporating solvent and diffusing nanoparticles. After introducing the model and its general behavior we focus on receding dewetting fronts which are initially straight but develop a transverse fingering instability. We analyze the dependence of the characteristics of the resulting branching patterns on the driving effective chemical potential, the mobility and concentration of the nanoparticles, and the interaction strength between liquid and nanoparticles. This allows us to understand the underlying instability mechanism.