Hybrid AI Models for Crypto Asset Trading and Risk Control Deep Learning and Reinforcement Learning in High-volatility Markets
Nguyen Tuan Phong
Le Quy Don Technical University, Vietnam.
Nguyen Vu Nguyen Khoi
Le Quy Don Technical University, Vietnam.
Doan Tien Ban
Le Quy Don Technical University, Vietnam.
Do Duy Nhat *
Le Quy Don Technical University, Vietnam.
Ngo Dai Phong
Le Quy Don Technical University, Vietnam.
Nguyen Hong Diep
Le Quy Don Technical University, Vietnam.
*Author to whom correspondence should be addressed.
Abstract
This paper proposes a hybrid artificial intelligence framework that integrates deep learning and reinforcement learning for cryptocurrency trading and risk control in highly volatile markets. The framework employs a Long Short-Term Memory (LSTM) network to forecast short-term asset returns and volatility, which are then incorporated into the state representation of a Proximal Policy Optimization (PPO) reinforcement learning agent. The RL agent learns trading policies that dynamically adjust portfolio positions by optimizing a risk-aware reward function that accounts for portfolio returns, Value-at-Risk (VaR), drawdown penalties, and transaction costs.
The proposed hybrid model is evaluated on multiple cryptocurrency assets, including Bitcoin, Ethereum, Binance Coin, and Solana, using historical market data spanning different market regimes. Experimental results demonstrate that the hybrid LSTM–PPO framework consistently outperforms deep learning–only, reinforcement learning–only, and rule-based baseline strategies in terms of risk-adjusted performance metrics such as the Sharpe ratio, maximum drawdown, and VaR compliance. These findings indicate that combining deep temporal forecasting with risk-aware reinforcement learning provides a robust and effective approach for autonomous trading and portfolio risk management in high-volatility cryptocurrency markets.
Keywords: Cryptocurrency trading, reinforcement learning, deep learning, portfolio risk control, PPO, Actor-Critic, market microstructure, transaction costs, sentiment analysis