Parsing data represented in hex - from a csv file

hi,

This is not from Dataquest exercises - i don’t know whether guidelines permit to answer apart from questions related to dataquest courses.

i have an example data from a csv file as shown below:
1st line tells the column names and 2nd line has contents.

i have extracted columns from D1 to D8. but i want to decode as these are in HEX and want them in string or whatever the actual data is encoded.

i used fromhex() function like : bytearray.fromhex().decode(‘utf-8’)
but unable to get the results. as i am only getting error as this is non-hexadecimal number.

after extracting (data D1 to D8) from csv file it was in type was str , which is the exact format required to use in fromhex() function. Then why is this not executing ?

TYPE,TIME,ID,D1,D2,D3,D4,D5,D6,D7,D8,ASCII,COUNT,PERIOD
RD11,4.2386,424,3,0,C,0,0,16,1,FF, ,107,0.0397
RD11,4.2394,375,F,F,0,0,0,0,0,0, ,43,0.1000
RD11,4.2396,236,10,BD,F,FF,20,0,0,8C, ,424,0.0100

The bytes.fromhex documentation says that the input “must contain two hexadecimal digits per byte”.

That’s why if you use only one digit, you get an error:

>>> bytes.fromhex("C")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: non-hexadecimal number found in fromhex() arg at position 1

However, if you append a 0 to the left of C

>>> bytes.fromhex("0C")
b'\x0c'

… great success!

The choice of 0 is not arbitrary. Mathematically 0 equals 0C in the same way that 0001337 equals 1337 — zeroes to the left are ignored.

You’ll probably find the method str.zfill useful:

What kind of data is this? It’s very strange encoding… Eight HEX values. I would think this is RGBA codes, but you type, time and count values… Any description to this data?

Hi Bruno,

Thank you for an early response!
i didn’t see this from python.org website, my bad … i was just using random blogs and experimenting.
you are right it takes in 2 digits per bytes.
i will try the soln and revert

1 Like

It is CAN bus data (Controller Area Network) used in automobiles.