That would most likely depend on where the property is located but the most common answer to that question is YES.
Property Taxes, Mortgages, Liens, and some other encumbrances are tied directly to the property, not the person that ownes it. For instance, the IRS has the right to place a tax lien on your property if you are delinquent on your taxes. That lien is then attached to the property. If you quit claim (quit is the actual term, not quick) your home to your neighbor, then the lien follows the property and your neighbor will have to pay that tax lien off if they try to sell the property.
Some good advice is to never obtain a property without first doing some research on it. At the very least, check the Taxing Authority, Water, Sewer, and any other entity that may have the right to lien the property. This will give you a general idea if there of the properties status.
HOWEVER, there may also be judgments and other liens attached to the property that you can’t find on your own and they will become a problem for you. They may not be your judgments but they will become your problem when you attempt to sell the property or get a mortgage on it.
It is HIGHLY advisable that you have a title company do a complete title search on the property BEFORE you have it deeded to you, it is well worth the expense.
Ultmately someone has to pay them. The County or State will not just forgive them. So, if you don’t have an agreement with the prior owner to pay the past due taxes that will stick, you are going to have to pay the delinquent taxes. This is one of the reasons there is an escrow or an attorney (depending on the state) that makes sure that all parties that have a claim on the property are paid. There is another type of lien that you could be responsible for called a “mechanics lien”. This is a lien that is put on a property when a vendor, like a building contractor, has not been paid. There can be other types or liens or easments that could affect your title and the seller may or may not know of them.
To do a transaction like this without professional help is like asking for problems down the road.