He was welcomed in the Eritrean capital Asmara by President Isaias Afewerki. The trip follows a visit to Addis Ababa by an Eritrean delegation.
Mr Abiy, who took office in April, has accepted a ruling awarding disputed territory to Eritrea.
Eritrea won independence from Ethiopia in 1993 after a lengthy conflict.
But within five years the neighbours were fighting over remote border areas, including the town of Badme.
Tens of thousands of people were killed in two years of fighting.
The land was awarded to Eritrea by a 2002 border commission ruling, but Ethiopia refused to accept it and retained its troops there.
The two sides have remained on a war footing since then.
- 24 May 1993: Eritrean independence from Ethiopia officially declared
- 6 May 1998: Border war begins
- 18 June 2000: Agreement on Cessation of Hostilities signed
- 12 December 2000: Algiers Peace Agreement signed
- 13 April 2002: The Eritrea-Ethiopia Boundary Commission delivered its “binding” ruling