Directional Feature Interaction Selection-Based Method for Structured Data Processing in Healthcare

doi:10.19678/j.issn.1000-3428.0070391

Abstract

Abstract:

Structured medical data are widely used in real-world healthcare systems. Owing to their unique characteristics, they typically exhibit properties such as being multi-source and heterogeneous with many potential candidate features, resulting in irregular data patterns that pose challenges to the feature-processing procedures of neural networks. Therefore, a recurrent directional feature processing network with a feature-embedding level similar to that of the Transformer architecture, named RDFP-NET, is proposed. The network possesses a favorable feature organization pattern and performs end-to-end processing directly on the collected raw medical data, enabling it to fully leverage effective information while eliminating the negative impact of unfavorable features. In addition, it employs a recurrent structure to correlate contextual feature information, further enhancing network task performance. RDFP-NET mainly comprises: a nonlinear feature-embedding method; a ranked feature interaction module that incorporates inductive bias into the general attention mechanism by calculating the information value of each feature and applying specific masking operations, achieving directional information flow and feature interaction among features with varying contribution values; a filtered feature selection module that discards redundant features in a filter-like manner by adjusting the Exponential Linear Unit (ELU) activation function; and a recurrent structure comprising two modules that facilitates good data fitting and contextual feature information flow while gradually addressing irregular feature patterns. Experiments conducted on five datasets encompassing two different target tasks demonstrate that RDFP-NET achieves an optimal average rank of 2.2 and a rank standard deviation of 0.75, showing excellent task performance and stability. Thus, RDFP-NET can provide medical professionals with more effective auxiliary information for clinical decision-making.

Key words: structured data, feature embedding, sorting interactive attention, feature selection, recurrent network

摘要：

医疗结构化数据在现实医疗体系中应用广泛, 由于其特殊性, 通常具有多源异构、存在大量潜在候选特征等性质, 表现出不规则的数据模式, 对神经网络的特征处理过程提出了挑战。因此, 提出一种类Transformer架构的特征嵌入级循环定向特征处理网络RDFP-NET, 其拥有良好的特征组织模式, 直接对采集的原始医疗数据进行端到端处理, 能够充分利用有效信息并摒弃不利特征的负面影响, 并通过循环结构关联上下文特征信息, 进一步改善网络任务性能。RDFP-NET主要包括: 一种非线性特征嵌入方法; 一个排序特征交互模块, 通过计算各特征的信息值和特定掩码操作, 在普通注意力机制中纳入归纳偏差, 实现不同贡献值特征间的定向信息流动和特征交互; 一种过滤特征选择模块, 通过对ELU激活函数的调整, 以类似过滤器的方式对冗余特征进行舍弃; 一个包括两种模块的循环结构, 实现良好的数据拟合和上下文特征信息流动, 并逐步处理特征的不规则模式。在包含两种不同目标任务的5个数据集上进行实验, 结果表明, RDFP-NET取得了2.2的最优平均排名和0.75的排名标准差, 显示出优秀的任务性能及稳定性, 能够为医疗工作者带来更有效的临床决策辅助信息。

关键词: 结构化数据, 特征嵌入, 排序交互注意力, 特征选择, 循环网络

WANG Weicheng, WAN Shuzhen, ZHANG Xiaolin, CHENG Zemin. Directional Feature Interaction Selection-Based Method for Structured Data Processing in Healthcare[J]. Computer Engineering, 2026, 52(7): 434-444.

王伟成, 万书振, 张晓磷, 程泽敏. 基于定向特征交互选择的医疗结构化数据处理方法[J]. 计算机工程, 2026, 52(7): 434-444.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0070391

https://www.ecice06.com/EN/Y2026/V52/I7/434

Figures/Tables 16

Fig.1 Schematic diagram of healthcare structured data

Fig.2 The process of the method in this paper

Fig.3 Data embedding process

Fig.4 The overall framework of RDFP-NET

Fig.5 Stacked bar chart of relative performance of each model on five datasets

Fig.6 Feature ranking results

Fig.7 The impact of L on the dataset

References 38

1	RYAN M . Deep learning with structured data.
2	ULMER D, MEIJERINK L, CINÀ G. Trust issues: uncertainty estimation does not enable reliable OOD detection on medical tabular data[EB/OL]. [2024-08-05]. https://arxiv.org/abs/2011.03274.
3	CLEMENTS J M, XU D, YOUSEFI N, et al. Sequential deep learning for credit risk monitoring with tabular financial data[EB/OL]. [2024-08-05]. https://arxiv.org/abs/2012.15330.
4	AHMED M , AFZAL H , MAJEED A , et al. A survey of evolution in predictive models and impacting factors in customer churn. Advances in Data Science and Adaptive Analysis, 2017, 9 (3): 1750007. doi: 10.1142/S2424922X17500073
5	ZHANG S , YAO L N , SUN A X , et al. Deep learning based recommender system: a survey and new perspectives. ACM Computing Surveys, 2020, 52 (1): 1- 38.
6	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[EB/OL]. [2024-08-05]. https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf.
7	LI Z W , LIU F , YANG W J , et al. A survey of convolutional neural networks: analysis, applications, and prospects. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33 (12): 6999- 7019. doi: 10.1109/TNNLS.2021.3084827
8	DONG L H, XU S, XU B. Speech-Transformer: a no-recurrence sequence-to-sequence model for speech recognition[C]//Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Washington D.C., USA: IEEE Press, 2018: 5884-5888.
9	GORISHNIY Y V, RUBACHEV I, KHRULKOV V, et al. Revisiting deep learning models for tabular data[EB/OL]. [2024-08-05]. https://arxiv.org/abs/2106.11959.
10	RAHAMAN N, BARATIN A, ARPIT D, et al. On the spectral bias of neural networks[EB/OL]. [2024-08-05]. https://arxiv.org/abs/1806.08734.
11	MITCHELL B R. The spatial inductive bias of deep learning[EB/OL]. [2024-08-05]. https://www.cs.montana.edu/sheppard/theses/mitchell-phd.pdf.
12	SHORTLIFFE E H, BARNETT G O. Medical data: their acquisition, storage, and use[EB/OL]. [2024-08-05]. https://link.springer.com/chapter/10.1007/978-0-387-21721-5_2.
13	GRINSZTAJN L, OYALLON E, VAROQUAUX G. Why do tree-based models still outperform deep learning on typical tabular data? [EB/OL]. [2024-08-05]. https://arxiv.org/abs/2207.08815.
14	ZHANG T R, ZHAO C J, CHEN G Y, et al. Feature contamination: neural networks learn uncorrelated features and fail to generalize[EB/OL]. [2024-08-05]. https://arxiv.org/abs/2406.03345.
15	KADRA A, LINDAUER M, HUTTER F, et al. Well-tuned simple nets excel on tabular datasets[EB/OL]. [2024-08-05]. https://arxiv.org/abs/2106.11189.
16	BALTRUŠAITIS T , AHUJA C , MORENCY L P . Multimodal machine learning: a survey and taxonomy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41 (2): 423- 443. doi: 10.1109/TPAMI.2018.2798607
17	CHEN T Q, GUESTRIN C. XGBoost: a scalable tree boosting system[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, USA: ACM Press, 2016: 785-794.
18	OSTROUMOVA L, GUSEV G, VOROBEV A, et al. CatBoost: unbiased boosting with categorical features[EB/OL]. [2024-08-05]. https://arxiv.org/abs/1706.09516.
19	SHWARTZ-ZIV R , ARMON A . Tabular data: deep learning is not all you need. Information Fusion, 2022, 81, 84- 90. doi: 10.1016/j.inffus.2021.11.011
20	KATZIR L, ELIDAN G, EL-YANIV R. Net-DNF: effective deep modeling of tabular data[EB/OL]. [2024-08-05]. https://openreview.net/forum?id=73WTGs96kho.
21	HAZIMEH H, PONOMAREVA N, MOL P, et al. The tree ensemble layer: differentiability meets conditional computation[EB/OL]. [2024-08-05]. https://arxiv.org/abs/2002.07772.
22	POPOV S, MOROZOV S, BABENKO A. Neural oblivious decision ensembles for deep learning on tabular data[EB/OL]. [2024-08-05]. https://arxiv.org/abs/1909.06312.
23	徐红, 矫桂娥, 张文俊, 等. 基于卷积神经网络的结构化非平衡数据分类算法. 计算机工程, 2023, 49 (2): 81- 89.
	XU H , JIAO G E , ZHANG W J , et al. Classification algorithm for structured imbalanced data based on convolutional neural network. Computer Engineering, 2023, 49 (2): 81- 89.
24	徐宛扬, 李文根, 关佶红. 面向金融网页数据的异构表格信息提取模型. 计算机应用, 2023, 43 (S1): 56- 60.
	XU W Y , LI W G , GUAN J H . Heterogeneous table information extraction model for financial web data. Journal of Computer Applications, 2023, 43 (S1): 56- 60.
25	WANG R X, SHIVANNA R, CHENG D, et al. DCN-V2: improved deep & cross network and practical lessons for web-scale learning to rank systems[C]//Proceedings of the Web Conference 2021. New York, USA: ACM Press, 2021: 1785-1797.
26	ARIK S Ö , PFISTER T . TabNet: attentive interpretable tabular learning. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35 (8): 6679- 6687. doi: 10.1609/aaai.v35i8.16826
27	SONG W P, SHI C C, XIAO Z P, et al. AutoInt: automatic feature interaction learning via self-attentive neural networks[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management. New York, USA: ACM Press, 2019: 1161-1170.
28	SOMEPALLI G, GOLDBLUM M, SCHWARZSCHILD A, et al. SAINT: improved neural networks for tabular data via row attention and contrastive pre-training[EB/OL]. [2024-08-05]. https://arxiv.org/abs/2106.01342.
29	张家豪. 基于深度学习的结构化数据分类技术研究[D]. 沈阳: 沈阳航空航天大学, 2023.
	ZHANG J H. Research on structured data classification technology based on deep learning[D]. Shenyang: Shenyang Aerospace University, 2023. (in Chinese)
30	HOLLMANN N, MVLLER S, EGGENSPERGER K, et al. TabPFN: a Transformer that solves small tabular classification problems in a second[EB/OL]. [2024-08-05]. https://arxiv.org/abs/2207.01848.
31	PEDREGOSA F, VAROQUAUX G, GRAMFORT A, et al. Scikit-learn: machine learning in Python[EB/OL]. [2024-08-05]. https://arxiv.org/abs/1201.0490.
32	RESHEF D N , RESHEF Y A , FINUCANE H K , et al. Detecting novel associations in large data sets. Science, 2011, 334 (6062): 1518- 1524.
33	CLEVERT D A, UNTERTHINER T, HOCHREITER S. Fast and accurate deep network learning by Exponential Linear Units (ELUs) [EB/OL]. [2024-08-05]. https://arxiv.org/abs/1511.07289.
34	LOSHCHILOV I, HUTTER F. Decoupled weight decay regularization[EB/OL]. [2024-08-05]. https://arxiv.org/abs/1711.05101.
35	Liver Disorders. UCI machine learning repository[EB/OL]. [2024-08-05]. https://doi.org/10.24432/C54G67.
36	SHEDDEN K , TAYLOR J M G , ENKEMANN S A , et al. Gene expression-based survival prediction in lung adenocarcinoma: a multi-site, blinded validation study. Nature Medicine, 2008, 14 (8): 822- 827. doi: 10.1038/nm.1790
37	FERNANDES K, CARDOSO J S, FERNANDES J. Transfer learning with partial observability applied to cervical cancer screening[EB/OL]. [2024-08-05]. https://link.springer.com/chapter/10.1007/978-3-319-58838-4_27.
38	YAMAN O , ERTAM F , TUNCER T . Automated Parkinson's disease recognition based on statistical pooling method using acoustic features. Medical Hypotheses, 2020, 135, 109483. doi: 10.1016/j.mehy.2019.109483

[1]	WEI Haodong, WAN Liang. Android Malware Detection Method Based on Static Feature Combination in Graph Neural Networks [J]. Computer Engineering, 2026, 52(3): 190-200.
[2]	WU Guoxia, QIU Yaru, JIANG Feng. Dynamic Firefly Algorithm for Feature Selection Based on Neighborhood Granularity Conditional Entropy [J]. Computer Engineering, 2026, 52(1): 144-153.
[3]	ZHOU Sha, CHE Shengbing, KAO Youchen, ZHANG Xu, GUO Shenyi. Network Intrusion Detection Based on Feature Selection and Spatio-Temporal Features [J]. Computer Engineering, 2025, 51(7): 223-231.
[4]	MA Siyuan, JIANG Lin, LI Chunlin, HU Qintai, WU Jigang. Palmprint Recognition Method Based on Gabor Filter and Improved Linear Discriminant Analysis [J]. Computer Engineering, 2025, 51(6): 320-326.
[5]	LIU Wenkai, LING Qinghua, WANG Zhichao. Two-stage Multi-modal Multi-objective Particle Swarm Optimization Feature Selection Algorithm Based on Decision Space Diversity Enhancement [J]. Computer Engineering, 2025, 51(12): 171-179.
[6]	LI Junyi, LI Xiangyang, LONG Chaoxun, LI Haiyan, LI Hongsong, YU Pengfei. Wild Mushroom Classification Based on Multi-level Region Selection and Cross-layer Feature Fusion [J]. Computer Engineering, 2024, 50(9): 179-188.
[7]	ZHANG Jian, ZHANG Bo. Biological Invasion-Based Feature Selection Algorithm [J]. Computer Engineering, 2024, 50(9): 46-53.
[8]	LIU Zhongmin, YAN Liang. Inpainting Model of Dunhuang Mural Fusing Dynamic Feature and Attention [J]. Computer Engineering, 2024, 50(5): 342-353.
[9]	Jie ZHAO, Wenhao YE, Zhouyang LIANG, Jianxin CHEN, Zhenning DONG. Fuzzy Rough Set Feature Selection Based on Inconsistent Nearest Neighbors [J]. Computer Engineering, 2024, 50(1): 110-119.
[10]	Xuan YANG, Jianmin MA, Manjun ZHAO. Feature Selection of High-Dimensional Time-Series Data Based on Neighborhood Mutual Information [J]. Computer Engineering, 2023, 49(7): 135-142.
[11]	WANG Xindi, YANG Su, ZHANG Siyuan, LUO Wuyang, LI Jie, LIU Hui. Urban Fire Risk Prediction Based on Spatial-Temporal Big Data and Satellite Images [J]. Computer Engineering, 2023, 49(6): 242-249.
[12]	XU Hong, JIAO Guie, ZHANG Wenjun, CHEN Yimin. Classification Algorithm for Structured Imbalanced Data Based on Convolutional Neural Network [J]. Computer Engineering, 2023, 49(2): 81-89.
[13]	LEI Lixiang, WU Zhihao, LIU Yu, ZHOU Zizhan. Click-Through Rate Prediction and Optimization Based on Intra-Field Features Similarity [J]. Computer Engineering, 2023, 49(2): 238-245.
[14]	LIU Li, ZHANG Desheng, XIAO Yanting. Fuzzy Weighted k-Nearest Centroid Neighbor Algorithm Based on Membership [J]. Computer Engineering, 2022, 48(7): 122-129.
[15]	AI Chenghao, GAO Jianhua, HUANG Zijie. Code Smell Detection Driven by Hybrid Feature Selection and Ensemble Learning [J]. Computer Engineering, 2022, 48(7): 168-176,198.

Please choose a citation manager

Content to export