Can I build a passive house with hempcrete? 

Yes. You may wish to combine the hempcrete with other types of natural insulation such as wood fibre board to reduce the wall thickness somewhat. However, the hempcrete element, making up the bulk of the wall thickness, will provide greater thermal mass than most other natural insulations, and will increase the thermal performance of the building.