Parsing data represented in hex - from a csv file

Q&A DQ Courses
#1

hi,

This is not from Dataquest exercises - i don’t know whether guidelines permit to answer apart from questions related to dataquest courses.

i have an example data from a csv file as shown below:
1st line tells the column names and 2nd line has contents.

i have extracted columns from D1 to D8. but i want to decode as these are in HEX and want them in string or whatever the actual data is encoded.

i used fromhex() function like : bytearray.fromhex().decode(‘utf-8’)
but unable to get the results. as i am only getting error as this is non-hexadecimal number.

after extracting (data D1 to D8) from csv file it was in type was str , which is the exact format required to use in fromhex() function. Then why is this not executing ?

TYPE,TIME,ID,D1,D2,D3,D4,D5,D6,D7,D8,ASCII,COUNT,PERIOD
RD11,4.2386,424,3,0,C,0,0,16,1,FF, ,107,0.0397
RD11,4.2394,375,F,F,0,0,0,0,0,0, ,43,0.1000
RD11,4.2396,236,10,BD,F,FF,20,0,0,8C, ,424,0.0100

#2

The bytes.fromhex documentation says that the input “must contain two hexadecimal digits per byte”.

That’s why if you use only one digit, you get an error:

>>> bytes.fromhex("C")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: non-hexadecimal number found in fromhex() arg at position 1

However, if you append a 0 to the left of C…

>>> bytes.fromhex("0C")
b'\x0c'

… great success!

The choice of 0 is not arbitrary. Mathematically 0 equals 0C in the same way that 0001337 equals 1337 — zeroes to the left are ignored.

You’ll probably find the method str.zfill useful:

image
image.png834×241 18.7 KB

How can I influence the sequence of a for loop (and how should I know?)
#3

What kind of data is this? It’s very strange encoding… Eight HEX values. I would think this is RGBA codes, but you type, time and count values… Any description to this data?

#4

Hi Bruno,

Thank you for an early response!
i didn’t see this from python.org website, my bad … i was just using random blogs and experimenting.
you are right it takes in 2 digits per bytes.
i will try the soln and revert

1 Like
#5

It is CAN bus data (Controller Area Network) used in automobiles.