Enhancing real estate prediction with entropy-based pattern analysis and economic sentiment integration

Abstract

Purpose The purpose of this research is to improve real estate market forecasting by introducing an innovative entropy-based multivariate clustering technique, Jensen–Shannon Time Series Segmentation (JSTS). The approach is designed to be integrated with advanced deep learning models such as Transformer, TSMixer, N-HiTS and LSTM, while incorporating key economic features such as consumer price index (CPI), sentiment analysis from headline news and real estate price and volume data to capture complex market dynamics. Design/methodology/approach This study applies the JSTS clustering technique on a real dataset comprising Australian real estate market data, including property prices, volumes, CPI and sentiment scores derived from headline news. Advanced deep learning models such as Transformer, TSMixer, N-HiTS and LSTM are used for forecasting, and performance is compared against traditional clustering methods like weighted dynamic time warping (WDTW) and standalone forecasting models such as Autoregressive Integrated Moving Average (ARIMA). The models are evaluated based on forecasting accuracy metrics, including MAPE, RMSE and MASE. Findings The experimental results show that the integration of JSTS clustering with advanced forecasting models significantly enhances the prediction accuracy compared to both traditional clustering methods and standalone forecasting models like ARIMA. The inclusion of sentiment from headline news further improves forecasting by providing real-time market sentiment insights, enabling the models to better capture market trends and behavioral shifts. Research limitations/implications While the proposed JSTS clustering method significantly improves forecasting accuracy, there are limitations in its computational complexity, especially when applied to large-scale, high-frequency datasets. The clustering process introduces additional computational overhead, which may require optimization for real-time applications. Further research is needed to explore the scalability of the JSTS approach across other domains such as stock markets or healthcare. Additionally, the model may require fine-tuning and domain expertise to adapt to varying market conditions, especially in different geographical regions. Practical implications The integration of sentiment analysis from headline news with economic indicators such as price, volume and CPI in real estate forecasting provides actionable insights for investors, developers and policymakers. By improving prediction accuracy, the proposed framework can support better decision-making in the real estate sector, helping stakeholders to anticipate market trends, assess risks and optimize investments. This methodology can also be adapted for use in other financial markets or sectors where time series data and sentiment analysis are key factors. Social implications Accurate forecasting of real estate markets has significant social implications, as it influences housing affordability, urban development and financial stability. By improving prediction models, this research can contribute to more informed policies regarding housing, economic planning and real estate investments. Additionally, incorporating sentiment analysis reflects the impact of public opinion and economic sentiment on market dynamics, potentially offering better tools for understanding market behaviors during periods of social or economic instability, such as during pandemics or financial crises. Originality/value This research presents a novel framework by combining JSTS clustering with state-of-the-art deep learning forecasting models and incorporating headline news sentiment into the prediction process. The method outperforms traditional approaches, offering a more accurate and robust solution for real estate time series forecasting, particularly in the volatile and complex Australian market. The integration of sentiment analysis into a multivariate framework is a key innovation, providing a deeper understanding of market behaviors.