Unexpected UnicodeEncodeError when saving text to a file in Python 3.10 with specific encoding
Hey everyone, I'm running into an issue that's driving me crazy. I've searched everywhere and can't find a clear answer. I'm working with a `UnicodeEncodeError` while trying to save a string to a file using UTF-16 encoding in Python 3.10. The string contains a mix of regular ASCII characters and some non-ASCII characters. Here's the relevant part of my code: ```python text = 'Hello, world! ΠΡΠΈΠ²Π΅Ρ, ΠΌΠΈΡ!' with open('output.txt', 'w', encoding='utf-16') as f: f.write(text) ``` When I run this, I get the following behavior: ``` UnicodeEncodeError: 'utf-16' codec need to encode character '\u043f' in position 13: surrogates not allowed ``` I tried changing the `encoding` to `utf-8` and it worked fine, but I need the file specifically in UTF-16 due to compatibility with a legacy system. I also considered using `utf-16-le` but that does not seem to resolve the scenario either. I've checked my string for any special characters and it looks fine. How can I resolve this behavior while keeping the encoding as UTF-16? What am I doing wrong? Any advice would be much appreciated.