Financial talk

My coworker and I are both administrative assistants to our directors. I work really hard to make improvements in my department and make things better. My coworker messes around and does none work related things for hours a day. Then comes and talks to me about it. It drives me crazy bc our departments are closely related and I end up working overdrive bc he doesnt do work things. I have talked to his director about this a little and nothing seems to have changed. It actually makes me want to leave and I feel like no one else sees that we are wasting out money! What can I do? Since our departments are closely linked, we share some responsibilities. Since he doddles and wastes time, I end up doing all the work.....