Hello,
This is not currently a built-in feature, but you can implement the functionality with a little custom code. You can either use custom JavaScript or PHP. A user shared a JavaScript example that worked for them:
http://formidablepro.com/help-desk/limit-checkbox-selection-3/
Custom JavaScript can go in your form's Settings > Customize HTML in the Before Fields.
Or, if you choose to use PHP, you can use this hook:
http://formidablepro.com/knowledgebase/frm_validate_field_entry/
Custom PHP can go in your theme functions.php or a new plugin that holds custom code.
You can also vote for this feature here:
https://formidablepro.com/help-desk/limit-checkbox-selections/
Our support does not cover writing or debugging custom code so you will need to complete this customization on your own, but if you have any non-code questions about this, feel free to post in our help desk:
https://formidablepro.com/help-topics/