By Dion Nissenbaum

ISTANBUL—The U.S. and Turkey have reached an understanding meant to assure the Ankara government that plans to drive Islamic State militants from a proposed safe zone in northern Syria won’t clear the way for Kurdish fighters to move in.

Officials from both countries on Monday disclosed details of the discussions that grew out of Turkey’s recent decision to step up its cooperation with the U.S. in the fight against Islamic State.

The U.S.-led coalition fighting the extremist group has conducted numerous airstrikes over the past year to back the Kurdish YPG militia in northern Syria, which has proved to be the most effective ground force fighting Islamic State.

But the U.S.-allied Turkish government is embroiled in a decades-old conflict with its own Kurdish minority. Turkey has resisted working with the YPG out of concern that the militants are laying the groundwork for the creation of a new Kurdish nation along Syria’s northern border with Turkey.

Two weeks ago, Turkey agreed to launch airstrikes targeting Islamic State fighters in Syria and allow the U.S. to use bases on its soil for the first time to do the same. At Turkey’s urging, the U.S. agreed to use airstrikes to protect a border zone free of Islamic State and controlled by moderate Syrian rebels.

Before Syria’s war erupted four years ago, the country’s Kurds were concentrated in three enclaves spread along the northern border. Over the past year, they have risen up to beat back advancing Islamic State fighters, most notably in the border town of Kobani.

The YPG advances have allowed Kurdish forces to establish authority over more Syrian territory than before the war, according to the Institute for the Study of War, which tracks control of land in the fight against Islamic State. In recent months, backed by U.S. airstrikes, the YPG has forced Islamic State fighters out of 2,000 square miles of territory in northern Syria—an area the size of Delaware—according to the U.S. military.

Since regime forces withdrew from Kurdish areas, the Syrian Kurds have secured a degree of newfound autonomy that has fueled aspirations for independence across the region. They have set up their own administration and defense forces that have started taking responsibility for security in the three Kurdish cantons. The YPG victory over Islamic State in the town of Tal Abyad this summer established a physical link between two of the three Kurdish cantons for the first time.

The Syrian Kurdish militia has pushed toward the eastern banks of the Euphrates River, the edge of Islamic State-controlled areas on the other side. The border zone the U.S. and Turkey want to set up is on the western side of the river.

YPG leaders said Monday they would work closely with allies, including the U.S.-led coalition and moderate rebel forces such as the Free Syrian Army or FSA, in the fight against Islamic State—also known as ISIS or ISIL.

However, they said they had made no commitment not to cross the Euphrates.

“The initial plan is to move to liberate the western side of the Euphrates once the areas to the east have been cleared of ISIS,” said Idres Nassan, a senior Kurdish official in Kobani. “But the YPG is acting in coordination with the local groups, such as the FSA and other groups fighting ISIS, as well as the coalition members.”

Preventing Kurdish forces from taking advantage of U.S. and Turkish airstrikes in the area is “red line” for Turkey as it steps up to play a greater role in battling Islamic State, a Turkish official said Monday.

Keeping Kurdish fighters from moving farther west restricts America’s ability to work in northwestern Syria with a Kurdish militia that has proved an effective fighting force. And it puts more pressure on the U.S. and Turkey to find an alternative capable of filling the void.

The challenges the anti-Islamic State coalition faces in developing that fighting force became clear last week when al Qaeda-linked militants attacked U.S.-backed rebels in Syria. The Nusra Front ambushed American-trained fighters, abducted some of their top leaders and vowed to prevent the group from rising up as an effective rival.

In response to the attack, the U.S.-led coalition carried out airstrikes to prevent the American-trained rebels in Syria from being overrun. The battles marked an early setback for the Pentagon’s fledgling efforts to create a reliable new ally in Syria to match the effectiveness of Kurdish forces.

In parallel with its new military strikes against Islamic State, Turkey has targeted bases in northern Iraq used by the outlawed Kurdish separatist group PKK.

The deadly airstrikes came in response to increased attacks by the PKK against Turkish security forces that are threatening a fragile peace process.

While the PKK is closely aligned with the YPG in Syria, Turkey and the U.S. have classified the former as a terrorist group and the latter as key allies in the fight against Islamic State.

The area where Turkey hopes to establish the border zone is filled with ethnic Turkmen and Arabs and Turkish leaders fear that the Kurdish fighters will try to drive them out.

“That’s a red line,” said one Turkish official. “There are almost no Kurds in the area that would be the ISIL-free zone. Forcing the issue would trigger a new wave of ethnic cleansing, which is unacceptable to us.”

U.S. officials have offered Turkey reassurances that they won’t rely on the YPG in that area, but have sought to give themselves wiggle room to work with the Kurdish fighters in that area if the needs arise.

“We have an agreement on the limits of the areas of operations,” said one senior U.S. official. “But the Turks are not unhappy that the YPG have taken much of the border.”

The Turkish official said the government views the YPG as “more pragmatic and realistic” than the PKK.

But he sent a blunt warning to the YPG not to challenge that red line.

“They saw how Turkey reacted to the PKK terrorist attacks,” the Turkish official said. “They saw how Turkey responded to its own red lines. That should be some kind of deterrence for the YPG.”

—Ayla Albayrak contributed to this article from Istanbul.

The Wall Street Journal