The range of new machine learning number formats, combined with the hundreds already present within GPUs has highlighted challenges in specifying, implementing, and verifying numerical hardware. Architects need to explore multiple number formats and precisions, not easily done within standard modelling languages such as C/C++. We present a language, FPCore, that can represent any number format and precision in its purest form and associated simulators capable of comparisons against infinite precision results and exploring multi-precision design spaces. We provide a tool chain that automatically generates a bit accurate C-model from an FPCore representation. Such behavioral models, when converted to RTL, typically exhibit poor power, performance and area. The optimization of such models is beyond current industrial high-level synthesis tool capabilities. We present a tool specifically targeted at optimizing such datapath designs, generating competitive RTL.