High-contrast study of the candidate planets and protoplanetary disk around HD 100546

The nearby Herbig Be star HD 100546 is known to be a laboratory for the study of protoplanets and their relation with the circumstellar disk, which is carved by at least two gaps. We observed the HD 100546 environment with high-contrast imaging exploiting several different observing modes of SPHERE, including data sets with and without coronagraphs, dual band imaging, integral field spectroscopy and polarimetry. The picture emerging from these different data sets is complex. Flux-conservative algorithm images clearly show the disk up to 200 au. More aggressive algorithms reveal several rings and warped arms that are seen overlapping the main disk. Some of these structures are found to lie at considerable height over the disk mid-plane at about 30 au. Our images demonstrate that the brightest wings close to the star in the near side of the disk are a unique structure, corresponding to the outer edge of the intermediate disk at ~ 40 au. Modeling of the scattered light from the disk with a geometrical algorithm reveals that a moderately thin structure (H∕r = 0.18 at 40 au) can well reproduce the light distribution in the flux-conservative images. We suggest that the gap between 44 and 113 au spans between the 1:2 and 3:2 resonance orbits of a massive body located at ~ 70 au, which mightcoincide with the candidate planet HD 100546b detected with previous thermal infrared (IR) observations. In this picture, the two wings can be the near side of a ring formed by disk material brought out of the disk at the 1:2 resonance with the same massive object. While we find no clear evidence confirming detection of the planet candidate HD 100546c in our data, we find a diffuse emission close to the expected position of HD 100546b. This source can be described as an extremely reddened substellar object surrounded by a dust cloud or its circumplanetary disk. Its astrometry is broadly consistent with a circular orbital motion on the disk plane, a result that could be confirmed with new observations. Further observations at various wavelengths are required to fully understand the complex phenomenology of HD 100546.