Not sure of the reason for needing to remove the quick ability to clear the input, but add this to your widget's CSS -

.select2-container.select2-allowclear .select2-choice abbr{
  display: none !important;
						

Probably a bit late but if you add the following to your "settings" in the record-picker setup it will remove the ability to clear.

options="{allowClear: false}"
Platform Analytics – Removing auto-generated live link and reference column hyperlink in Developer forum How to remove null filed in approval email in Developer forum Un able to remove two empty relatedLists tabs in the UI Builder in Developer forum Insufficient Access for query_match and query_range on sys_user_grmember in Test Instance in Developer forum