Hexadecimal value 0x1F is an invalid character

On Sitecore 9.2.0 while rebuilding the Web or other publishing target Solr index We were facing a very intermittent issue. The rebuilding process was terminated in between and throwing the following exception:

Message: '◻', hexadecimal value 0x1F, is an invalid character.
Source: System.Xml
   at System.Xml.XmlEncodedRawTextWriter.WriteElementTextBlock(Char* pSrc, Char* pSrcEnd)
   at System.Xml.XmlEncodedRawTextWriter.WriteString(String text)
   at System.Xml.XmlWellFormedWriter.WriteString(String text)
   at System.Xml.Linq.ElementWriter.WriteElement(XElement e)
   at System.Xml.Linq.XElement.WriteTo(XmlWriter writer)
   at System.Xml.Linq.XNode.GetXmlString(SaveOptions o)
   at SolrNet.Commands.AddCommand`1.ConvertToXml()
   at SolrNet.Commands.AddCommand`1.Execute(ISolrConnection connection)
   at SolrNet.Impl.LowLevelSolrServer.SendAndParseHeader(ISolrCommand cmd)
   at Sitecore.ContentSearch.SolrProvider.SolrBatchUpdateContext.AddRange(IEnumerable`1 group, Int32 groupSize)
   at Sitecore.ContentSearch.SolrProvider.SolrBatchUpdateContext.Commit()
   at Sitecore.ContentSearch.AbstractSearchIndex.PerformUpdate(IEnumerable`1 indexableInfo, IndexingOptions indexingOptions)
   at Sitecore.ContentSearch.AbstractSearchIndex.Update(IEnumerable`1 indexableInfo)

The exception or crawling log or the content search logs doesn’t provide any specific item path which can help to figure out which specific item it is throwing the error for. Our Sitecore imports content from multiple outbound systems and create related items in Sitecore hence it is impossible to figure out which item or items are the culprit.

While we rebuild the Solr index, the index documents are submitted to a Solr server in XML format. Third-party media content extractors might produce characters that cannot be converted to XML. As a result, the entire documents batch cannot be indexed. Errors similar to the above can be found in log records.

During our troubleshooting and research we landed on a Sitecore KB Article which provides solutions about the exact similar issue according to the Sitecore Version you are on.

Since we are on Sitecore 9.2.0, we followed the steps below along with the patch installation and we are not seeing the exception any more since then.

1. Verify that the Sitecore.ContentSearch.SolrProvider.dll assembly version matches 5.0.0-r00290 (right-click on the file, click Properties, then Details, check the Product version property).
2. Download and install the following hotfix: SC Hotfix 380817-1

If you are on a different Sitecore version please find the details on this KB Article and follow the steps mentioned for your Sitecore version. If that doesn’t help, as suggested, please reach out to the Sitecore Support team.

Hope this helps!!!