Paying off your car will not lower your credit rating, nor will you see any penalties for doing so. Taking a good chunk out of your car payment can lessen the interest you will pay for the remainder of the note.
What is your highest interest rate for your debts? Credit cards, car, school...