Put the potatoes in a large pot of salted water and bring to a boil. Cook over moderately high heat until tender, about 25 minutes. Drain the potatoes, return to the pot and shake over high heat to dry them, about 30 seconds. Partially mash the potatoes with a fork, blending in the milk and butter. Season with salt. Let cool slightly, then shape the mashed potatoes into eight 1 1/2-inch-thick patties.
In a large skillet, heat 1/4 inch of oil until shimmering. Add 4 of the potato cakes and cook over moderate heat until browned and crusty, about 4 minutes per side. Transfer the cakes to a warm oven while you fry the rest. Serve hot.