基于深度高斯过程的多元类别数据分布估计

doi:10.19678/j.issn.1000-3428.0049671

摘要/Abstract

摘要：

多元类别数据的可能取值会随向量长度的增长呈指数级增长,从而造成数据稀疏性问题。通过将观察数据嵌入到连续空间中训练识别数据之间的相似性,构建多元类别数据的线性高斯隐变量模型和类别隐高斯过程(CLGP)。在CLGP模型基础上,建立小样本多元类别数据分布估计的多元类别深度隐高斯过程模型,并结合蒙特卡洛采样的变分推断方法对模型进行参数优化。实验结果表明,与CLGP模型相比,该模型分布估计精确度有所提升。

关键词: 多元类别数据, 生成式模型, 深度高斯过程, 无监督学习, 变分推断

Abstract:

The possible value of multivariate categorical data increases exponentially with the length of the vector,resulting in data sparsity.The similarity between the identified data is trained by embedding the observation data into the continuous space,and the linear Gaussian hidden variable model and the Categorical Latent Gaussian Process (CLGP) of the multi-category data are constructed.Based on the CLGP model,a multi-class deep hidden Gaussian process model for small sample multi-class data distribution estimation is proposed,and the parameters are optimized by Monte Carlo sampling.Experimental results show that compared with the CLGP model,this model distribution estimation accuracy has improved.

Key words: multivariate categorical data, generative model, Deep Gaussian Process(DGP), unsupervised learning, variational inference

中图分类号:

TP181

刘姝君,李艳婷. 基于深度高斯过程的多元类别数据分布估计[J]. 计算机工程, 2019, 45(2): 160-166.

LIU Shujun,LI Yanting. Multivariate Categorical Data Distribution Estimation Based on Deep Gaussian Process[J]. Computer Engineering, 2019, 45(2): 160-166.

https://www.ecice06.com/CN/Y2019/V45/I2/160

参考文献

［1］王盛玉,曾碧卿,胡翩翩.基于卷积神经网络参数优化的中文情感分析［J］.计算机工程,2017,43(8):200-207,214.
［2］BENGIO Y,SCHWENK H,SENCAL J S,et al.Neural probabilistic language models［M］.In Innovations in Machine Learning.Berlin,Germany:Springer,2006:137-186.
［3］COLLOBERT R,WESTON J.A unified architecture for natural language processing:deep neural networks with multitask learning［C］//Proceedings of the 25th International Conference on Machine learning.New York,USA:ACM Press,2008:160-167.
［4］GAL Y,CHEN Y,ZOUBIN G.Latent Gaussian process for distribution estimation of multivariate categorical data［EB/OL］.［2017-11-18］.https://arxiv.org/pdf/150 3.02182.pdf.
［5］KHAN M E,MOHAMED S,MARLIN B R,et al.A stick-breaking likelihood for categorical data analysis with latent Gaussian models［EB/OL］.［2017-11-18］.https://www.shakirm.com/papers/catLGM-AIstats 20 12.pdf.
［6］何志昆,刘光斌,赵曦晶,等.高斯过程回归方法综述［J］.控制与决策,2013,28(8):1121-1129,1137.
［7］RASMUSSEN C E,WILLIAM K I.Gaussian process for machine learning［EB/OL］.［2017-11-18］.http://www.gaussianprocess.org/gpml/.
［8］KO J,FOX D.GP-Bayes filters:Bayesian filtering using Gaussian process prediction and observation models［C］//Proceedings of IEEE/RSJ Intelligent Robots and Systems.Washington D.C.,USA:IEEE Press,2008:3471-3476.
［9］DEISENROTH M P,RASMUSSEN C E.PILCO:a model-based and data-efficient approach to policy search［C］//Proceedings of the 28th International Conference on Machine Learning.［S.l.］:Omnipress,2011:465-472.
［10］CRESSIE N,WIKLE K.Statistics for spatio-temporal data［EB/OL］.［2017-11-18］.https://www.wiley.com/en-us/Statistics+for+Spatio+Temporal+Data-p-9780471692744.
［11］王鑫,李红丽.台风最大风速预测的高斯过程回归模型［J］.计算机应用研究,2015,32(1):59-62.
［12］BRIOL F X,OATES C J,GIROLAMI M,et al.Probabilistic integration:a role for statisticians in numerical analysis?［EB/OL］.［2017-11-18］.https://arxiv.org/pdf/1512.00933v5.pdf.
［13］GUESTRIN C,KRAUSE A,SINGH A P.Near-optimal sensor placements in Gaussian processes［C］//Proceedings of the 22nd International Conference on Machine Learning.New York,USA:ACM Press,2005:265-272.
［14］孙晓燕,陈姗姗,巩敦卫,等.基于区间适应值交互式遗传算法的加权多输出高斯过程代理模型［J］.自动化学报,2014,40(2):172-184.
［15］SNOEK J,LAROCHELLE H,ADAMS R P.Practical bayesian optimization of machine learning algorithms［C］//Proceedings of the 25th International Conference on Neural Information Processing Systems［S.l.］:Curran Associates Inc.,2012:2951-2959.
［16］REZENDE D J,MOHAMED S,WIERSTRA D.Stochastic back propagation and approximate inference in deep generative models［J］.Pattern Recognition and Machine Learning,2014,32(2):1278-1286.
［17］GHAHRAMANI Z.Probabilistic machine learning and artificial intelligence［J］.Nature,2015,521:452-459.
［18］DAMIANOU A C,LAWRENCE N D.Deep Gaussian processes［EB/OL］.［2017-11-18］.https://core.ac.uk/download/pdf/46564399.pdf.
［19］WILSON A G,HU Z,SALAKHUTDINOV R,et al.Deep kernel learning［EB/OL］.［2017-11-18］.https://arxiv.org/pdf/1511.02222.pdf.
［20］DURRANDE N,GINSBOURGER D,ROUSTANT O.Additive kernels for Gaussian process modeling［EB/OL］.［2017-11-18］.https://arxiv.org/pdf/1103.4023.pdf.
［21］DAVID D,JAMES R L,ROGER G,et al.Structure discovery in nonparametric regression through compositional kernel search［EB/OL］.［2017-11-18］.http://www.cs.toronto.edu/~rgrosse/icml2013-gp.pdf.
［22］HENSMAN J,LAWRENCE N D.Nested variational compression in deep Gaussian processes［EB/OL］.［2017-11-18］.https://arxiv.org/pdf/1412.1370.pdf.
［23］VAFA K.Training deep Gaussian processes with sampling［EB/OL］.［2017-11-18］.http://approxi mateinference.org/accepted/Vafa2016.pdf.
［24］WANG Y,BRUBAKER M,CHAIB-DRAA B,et al.Sequential inference for deep Gaussian process［EB/OL］.［2017-11-18］.http://proceedings.mlr.press/v51/wang 16c.pdf.
［25］BUI T D,HERNNDEZ-LOBATO D,LI Y,et al.Deep Gaussian processes for regression using approximate expectation propagation［EB/OL］.［2017-11-18］.http://proceedings.mlr.press/v48/bui16.pdf.
［26］DAI Z,DAMIANOU A,GONZLEZ J,et al.Variational auto-encoded deep Gaussian processes［EB/OL］.［2017-11-18］.https://arxiv.org/pdf/1511.06455.pdf.
［27］DAMIANOU A D,TITSIAS M K,LAWRENCE N D.Variational Gaussian process dynamical systems［EB/OL］.［2017-11-18］.http://papers.nips.cc/paper/4330-variational-gaussian-process-dynamical-systems.pdf.
［28］CUTAJAR K,BONILLA E V,MICHIARDI P,et al.Practical learning of deep Gaussian processes via random fourier features［EB/OL］.［2017-11-18］.https://pdfs.semanticscholar.org/bafa/7e2d586e7bfe77d9a55ac1cff4 eb2f6ff292.pdf.
［29］HUGH S,MARC D.Doubly stochastic variational inference for deep Gaussian processes［EB/OL］.［2017-11-18］.https://arxiv.org/pdf/1705.08933.pdf.
［30］TITSIAS M K.Variational learning of inducing variables in sparse Gaussian processes［EB/OL］.［2017-11-18］.http://proceedings.mlr.press/v5/titsias09a/titsias09a.pdf.
［31］BLEI D M,KUCUKELBIR A,MCAULIFFE J D.Variational inference:a review for statisticians［EB/OL］.［2017-11-18］.https://arxiv.org/pdf/1601.00670.pdf.
［32］JOHN D,ELAD H,YORAM S.Adaptive subgradient methods for online learning and stochastic optimiza-tion［J］.Journal of Machine Learning Research,2011,12:2121-2159.
［33］TIELEMAN T,HINTON G.Lecture 6.5- rmsprop,COURSERA:neural networks for machine learning［EB/OL］.［2017-11-18］.http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf.
［34］TOM S,IOANNIS A,DAVID S.Unit tests for stochastic optimization［EB/OL］.［2017-11-18］.https://arxiv.org/pdf/1312.6055.pdf.
［35］Perplexity［G/OL］.［2017-11-18］.https://en.wiki pedia.org/wiki/Perplexity.
［36］Tensorflow［EB/OL］.［2017-11-18］.https://tenso rflow.google.cn/.
［37］MATTHEWS A G D G,MARK V D W,NICKSON T,et al.GPflow:a gaussian process library using tensorflow［EB/OL］.［2017-11-18］.http://adsabs.harvard.edu/abs/2016arXiv161008733M.
［38］Handwritten digits［DB/OL］.［2017-11-18］.https://cs.nyu.edu/~roweis/data.html.
［39］MATJAZ Z,MILAN S.Breast cancer data set［DB/OL］.［2017-11-18］.http://archive.ics.uci.edu/ml/datasets/Breast+Cancer.

[1]	李永飞, 李铭洋, 常鑫, 曹可欣. 基于可解释性深度学习的物联网水质监测数据异常检测[J]. 计算机工程, 2024, 50(6): 179-187.
[2]	余长宏, 许孔豪, 张泽, 高明. 基于分割点改进孤立森林的网络入侵检测方法[J]. 计算机工程, 2024, 50(6): 148-156.
[3]	何银银, 胡静, 陈志泊, 张荣国. 融合门控变换机制和GAN的低光照图像增强方法[J]. 计算机工程, 2024, 50(2): 247-255.
[4]	王书朋, 何引弟. 融合特征注意力机制的非均匀光照图像增强算法[J]. 计算机工程, 2023, 49(8): 232-239.
[5]	何悦, 陈广胜, 景维鹏, 徐泽堃. 基于深度多相似性哈希方法的遥感图像检索[J]. 计算机工程, 2023, 49(2): 206-212.
[6]	蔡瑞初, 伍运金, 陈薇, 郝志峰. 面向多元时间序列的群体因果关系发现算法[J]. 计算机工程, 2023, 49(2): 127-135.
[7]	鄢宁, 李岳阳, 罗海驰. 基于块金字塔记忆模块的无监督异常检测[J]. 计算机工程, 2023, 49(12): 304-310.
[8]	唐佳敏, 韩华, 黄丽. 行人再识别中基于无监督学习的粗细粒度特征提取[J]. 计算机工程, 2022, 48(4): 269-275,283.
[9]	张显炀, 朱晓宇, 林浩申, 刘刚, 安喜彬. 基于高斯混合-变分自编码器的轨迹预测算法[J]. 计算机工程, 2020, 46(7): 50-57.
[10]	许春杰, 吴蒙, 杨立君. 一种基于分层聚合的分布式异常数据检测方案[J]. 计算机工程, 2020, 46(4): 213-219.
[11]	柴梦婷, 朱远平. 生成式对抗网络研究与应用进展[J]. 计算机工程, 2019, 45(9): 222-234.
[12]	陈健,唐俊遥,朱生光,周兆钊. 深度堆栈自编码网络在船舶重量估算中的应用[J]. 计算机工程, 2019, 45(5): 315-320.
[13]	吉培培,陈恳,郭春梅,李萌. 多相机网络中行人的全局最优匹配[J]. 计算机工程, 2017, 43(1): 268-273.
[14]	庞磊, 李寿山, 周国栋. 基于情绪知识的中文微博情感分类方法[J]. 计算机工程, 2012, 38(13): 156-158,162.
[15]	史天艺;李明禄. 基于维基百科的自动词义消歧方法[J]. 计算机工程, 2009, 35(18): 62-65.

选择文件类型/文献管理软件名称

选择包含的内容