The head of Nasdaq OMX Group Inc. (NDAQ) cast doubt Monday on the ability of U.S. regulators to carve out some bank trading activities as part of a broad overhaul of the financial system.

Bob Greifeld, chief executive of the transatlantic exchange operator, said it appears "impossible" to separate banks' proprietary trading activities from customer and hedging positions.

Greifeld's comments on a post-earnings conference call are the loudest call to date from the exchange industry for a rethink of the reform proposals made to the Obama administration by former Fed chairman Paul Volcker.

"I think they have to find a different path," said Greifeld, reflecting broader industry concern about a measure that would impact some of the exchange industry's key client bases.

Greifeld also cautioned that opportunities for regulatory arbitrage remain amid international efforts to toughen oversight, though he noted "a new level of cooperation" in the process.

His comments came as Nasdaq OMX beat expectations with a 23% rise in fourth-quarter profit, and continued the expansion of its derivatives unit.

It also stemmed some of the slide in the market share of its U.S. equities business, and plans to press on with a struggling pan-European share trading operation.

Greifeld is still looking to build dealer bank support for the European platform, which launched amid a flurry of rival offerings in late 2008 and has claimed just 1% of the market.

"With the benefits of hindsight, we would've led with a consortium-based effort," said Greifeld. "It seems an opportune time for us to do something differently."

Greifeld said in an interview that he would not rule out acquiring a rival European trading platform. The London Stock Exchange Group PLC (LSE.LN) in December bought a stake in the dealer-backed Turquoise platform.

The company also announced Monday that its INET trading system was rolled out on the company's seven Nordic and Baltic equity markets, driving a 10-fold increase in trade execution speed, according to Greifeld.

Greifeld continued to downplay concerns that Borse Dubai could sell its 28% stake in Nasdaq OMX. He said board members of Borse Dubai "have communicated to us in no uncertain terms that they're long-term holders of their position in Nasdaq OMX."

Borse Dubai holds the emirate's positions in Nasdaq Dubai and the Dubai Financial Market Co. (DFM.DFM), which are expected to merge after DFM moved to buy Nasdaq Dubai for $121 million in late December. The transaction left Nasdaq OMX with a 1% stake in DFM and an $82 non-cash impairment charge.

Nasdaq OMX reported a fourth-quarter profit of $43 million, or 20 cents a share, up from $35 million, or 17 cents a share, a year earlier. Net revenue fell to $369 million from $403 million a year earlier.

Its shares were recently down 2.9% at $18.25.

-By Jacob Bunge, Dow Jones Newswires; 212-416-2855; nathan.becker@dowjones.com

(Nathan Becker contributed to this article.)