This is the most common and frustrating issue while parsing a csv file in rails i.e. the encoding issue. So in ruby on rails i found a library which basically detects the Character encoding using ICU (dependency package) and the name of the library is CharlockHolmes. And than we have to add the encoding if it is not already there to parsing it successfully. Below is the code:
contents = File.read(filepath) # filepath is the path of the file
begin
detection = CharlockHolmes::EncodingDetector.detect(contents) # passing the contents for detection
encoding = Encoding.find(detection[:encoding]) # find the encoding from detection object
rescue => error
Rails.logger.error("#{error.class}: #{error.message}\n#{error.backtrace.join("\n")}")
encoding = Encoding::ISO_8859_1 # overwrite if encoding is nil
end
Roo::CSV.new(filepath, file_warning: :ignore, csv_options: {encoding: encoding}) # adding encoding parameter so that it will not give error
Note: You must install libicu using this command if you are using ubuntu (sudo apt-get install libicu-dev)
0 Comment(s)