Fix Hexl handling of coding-systems with BOM

Fix Hexl handling of coding-systems with BOM

* lisp/international/mule-cmds.el (encode-coding-char): If
CODING-SYSTEM produces BOM, remove the BOM bytes from the produced
byte sequence.  (Bug#48324)

* lisp/hexl.el (hexl-mode): Use bufferpos-to-filepos to convert
point to offset into the original file.
(hexl-mode-exit, hexl-maybe-dehexlify-buffer): Use
filepos-to-bufferpos to restore point in the original buffer.
(hexl-mode, hexl-insert-multibyte-char)
(hexl-self-insert-command, hexl-insert-hex-char)
(hexl-insert-decimal-char, hexl-insert-octal-char)
(hexl-find-file): Enhance the doc strings, mainly explaining the
complications of inserting multibyte characters.
(hexl-insert-multibyte-char): Don't treat CH as unibyte if the
coding-system isn't ASCII-compatible.  Don't treat null bytes as
multibyte.