arXiv:2502.09257v3 Announce Type: replace-cross 
Abstract: We study the problem of contextual combinatorial semi-bandits, where input contexts are mapped into subsets of size $m$ of a collection of $K$ possible actions. In each round, the learner observes the realized reward of the predicted actions. Motivated by prototypical applications of contextual bandits, we focus on the $s$-sparse regime where we assume that the sum of rewards is bounded by some value $s\ll K$. For example, in recommendation systems the number of products purchased by any customer is significantly smaller than the total number of available products. Our main result is for the $(\epsilon,\delta)$-PAC variant of the problem for which we design an algorithm that returns an $\epsilon$-optimal policy with high probability using a sample complexity of $\tilde{O}((poly(K/m)+sm/\epsilon^2) \log(|\Pi|/\delta))$ where $\Pi$ is the underlying (finite) class and $s$ is the sparsity parameter. This bound improves upon known bounds for combinatorial semi-bandits whenever $s\ll K$, and in the regime where $s=O(1)$, the leading terms in our bound match the corresponding full-information rates, implying that bandit feedback essentially comes at no cost. Our algorithm is also computationally efficient given access to an ERM oracle for $\Pi$. Our framework generalizes the list multiclass classification problem with bandit feedback, which can be seen as a special case with binary reward vectors. In the special case of single-label classification corresponding to $s=m=1$, we prove an $O((K^7+1/\epsilon^2)\log(|H|/\delta))$ sample complexity bound, which improves upon recent results in this scenario. Additionally, we consider the regret minimization setting where data can be generated adversarially, and establish a regret bound of $\tilde O(|\Pi|+\sqrt{smT\log |\Pi|})$, extending the result of Erez et al. (2024) who consider the simpler single label classification setting.

أظهرت دراسة حديثة حول شبه العصابات التوافقية السياقية تقدمًا كبيرًا في تحسين تعقيد العينة مع المكافآت النادرة. هذه الأبحاث مهمة لأنها تعزز كفاءة الخوارزميات المستخدمة في أنظمة التوصية، مما يسمح باتخاذ قرارات أفضل مع موارد أقل. من خلال التركيز على نظام $s$-sparse، حيث تكون مجموع المكافآت محدودة، يمكن أن تؤدي النتائج إلى تطبيقات أكثر فعالية في مجالات مختلفة، مما يجعلها تقدمًا ملحوظًا في تعلم الآلة.

Un estudio reciente sobre semi-bandits combinatorios contextuales ha logrado avances significativos en la mejora de la complejidad de muestra con recompensas escasas. Esta investigación es crucial ya que mejora la eficiencia de los algoritmos utilizados en sistemas de recomendación, permitiendo una mejor toma de decisiones con menos recursos. Al centrarse en el régimen $s$-sparse, donde la suma de recompensas está limitada, los hallazgos podrían conducir a aplicaciones más efectivas en varios campos, lo que representa un avance notable en el aprendizaje automático.

Une étude récente sur les semi-bandits combinatoires contextuels a réalisé des avancées significatives dans l'amélioration de la complexité d'échantillonnage avec des récompenses rares. Cette recherche est cruciale car elle améliore l'efficacité des algorithmes utilisés dans les systèmes de recommandation, permettant une meilleure prise de décision avec moins de ressources. En se concentrant sur le régime $s$-sparse, où la somme des récompenses est limitée, les résultats pourraient conduire à des applications plus efficaces dans divers domaines, ce qui en fait une avancée remarquable en apprentissage automatique.

A recent study on contextual combinatorial semi-bandits has made significant strides in improving sample complexity with sparse rewards. This research is crucial as it enhances the efficiency of algorithms used in recommendation systems, allowing for better decision-making with fewer resources. By focusing on the $s$-sparse regime, where the sum of rewards is limited, the findings could lead to more effective applications in various fields, making it a noteworthy advancement in machine learning.

From Contextual Combinatorial Semi-Bandits to Bandit List Classification: Improved Sample Complexity with Sparse Rewards

Was this article worth reading? Share it

LucidQuery AI

Spicy Ranks AI

Hypertune

SciSummary

Sellm

CodeSpaced

Ready to build your own newsroom?