There are a few things that could cause this issue. If the water line in the back of the unit is plastic, it will need to be switched out with copper tubing. A plastic (flexible) hose can create a taste problem. Usually, the main reason for ice to get an odor or taste bad is because of the air flow, inside of the refrigerator. Air is circulated throughout the refrigerator and it blows directly on the ice cubes. The air will pick up any odors inside the refrigerator and blow it directly on the cubes, which in turn absorb the odor. To correct this, the inside of the refrigerator needs to be cleaned. The best way to clean the inside of a refrigerator is, with a mild detergent and warm water.
If the same taste is detected in the new ice and you have a water filter on the refrigerator, I recommend that you try removing the water filter for about 1 week and allow chlorinated water to flow through the water system in your ice maker. In some situations, a water filter will remove chlorine and harmless cysts will develop in the water system of the refrigerator that will affect the taste of the ice. Running chlorinated city water through the refrigerator water system will kill the cysts and eliminate this problem. I assume that you are in the city where the water supply is chlorinated. If you are on a ground water well system in a rural area, the cysts may be harder to eliminate. I hope this helps.