Interactions between cancer cells and factors within the tumor microenvironment (mE) are essential for understanding tumor development. The spatial relationships between blood vessel cells and cancer cells, e.g. tumor initiating cells (TICs), are an important parameter. Accurate segmentation of blood vessel is necessary for the quantization of their spatial relationships. However, this remains an open problem due to uneven intensity and low signal to noise ratio (SNR). To overcome these challenges, we propose a novel approach that integrates an oriented hidden Markov random field model (Ori-HMRF) with local clustering. The local clustering delineates boundaries of blood vessel segments with low SNR. Then blood vessel segments are viewed as random variables in the Ori-HMRF and their spatial dependence is defined based on directional information. The Ori-HMRF model suppresses noise and generates accurate blood vessel segmentation results. Experimental validations were conducted on both normal mammary and breast cancer tissues.