If air is coming out of the vent in the oil cap the problem will be in the cylinder and rings. The piston rings are not sealing against the cylinder walls allowing the air to blow past and pressurize the crankcase. To repair this you will need to take the head and valve plate off. Then inspect the piston, ring, and cylinder. If the cylinder wall is still smooth, you should be able to get by with replacing the piston and rings. You can order the parts from the following link.
Craftsman air compressor parts
Please let me know if you need further assistance.
Have a great day!!