Ibrahim El Shar
About
I am a Senior Data Scientist at Walmart, based in California. I lead the design and deployment of reinforcement learning systems for real-world optimization problems at scale — including dynamic pricing for crowd-sourced delivery and task assignment for in-store operations. My work draws from both operations research and machine learning, with a focus on sequential decision-making, multi-agent coordination, and robust learning in high-stakes environments. Previously, I was a Machine Learning Scientist at Hitachi America, where I led initiatives involving reinforcement learning and Generative AI and Large Language Models (LLMs) for supply chain resilience. I hold a Ph.D. in Operations Research from the University of Pittsburgh and a master's degree in Financial Engineering and Operations Research from the American University of Beirut.
News
- Decemeber 2023: I will be at NeurIPS 2023 (New Orleans, Dec 10th - Dec 16th) to present our paper, "Weakly Coupled Deep Q-Networks". Join our poster session on Tuesday, Dec 12, 5:15 p.m. CST, Great Hall & Hall B1+B2 (level 1) Poster #216.
- November 2023: Excited to serve as a panelist for the panel "Navigating Today's Supply Chain" in the Supply Chain Expo that is organized by the University of Louisville and Western Kentucky University, on Nov 9, 2023. In the panel, I discussed the significance of reinforcement learning in optimizing the supply chain and highlighted the great potential of generative AI across the supply chain, while also addressing its current limitations.
- October 2023: I will be presenting my work on weakly coupled MDPs at INFORMS 2023 Annual meeting at Phoenix, AZ.
- September 2023: Our paper on "Weakly Coupled Deep Q-Networks" got accepted to NeurIPS 2023!
- August 2023: New paper on "Multi-Objective Reinforcement Learning for Sustainable Supply Chain Optimization" got accepted to IEEE International Conference on Automation Science and Engineering (CASE 2023)
- August 2023: I will be attending KDD 2023. Hit me up if you will be there too!
- August 2022: Our paper on "Deep Reinforcement Learning toward Robust Multi-echelon Supply Chain Inventory Optimization" got accepted to IEEE International Conference on Automation Science and Engineering (CASE 2022)
- March 2022: Submitted a new paper titled "Deep Reinforcement Learning toward Robust Multi-echelon Supply Chain Inventory Optimization" to IEEE International Conference on Automation Science and Engineering (CASE). This paper is based on my work during my internship at Hitachi America, Ltd. R&D.
- February 2022: Excited to join Amazon as a Research Scientist Intern!
- June 2021: Will start work as a research machine learning intern for Hitachi America LTD industrial AI lab!
- November 11, 2020: Will present my work at Virtual 2020 INFORMS Annual Meeting.
- July 14, 2020: Join our poster session at ICML 2020 to discuss LBQL!
- June 1, 2020: Our paper "Lookahead-bounded Q-learning" got accepted to ICML2020.
- October 20 - 23, 2019: At INFORMS Annual meeting 2019 to give a talk about Lookahead-Bounded QL.
Research Publications
Publications in peer-reviewed conferences and journals. My research focuses on reinforcement learning, operations research, and supply chain optimization.
Bound enhanced reinforcement learning system for distribution supply chain management
2024
Haiyan Wang, Atsuki Kiuchi, Hsiu-Khuern Tang, Chetan Gupta, EL-SHAR Ibrahim, SUN Wenhuan
US Patent App. 18/105,593
A novel system that combines bound-enhanced reinforcement learning techniques for optimizing distribution supply chain management, using upper and lower bounds of optimal inventory costs with epsilon-greedy exploration in multi-echelon supply chains.
Weakly Coupled Deep Q-Networks
2023
Ibrahim El Shar, Daniel R. Jiang
Advances in Neural Information Processing Systems 37 (NeurIPS 2023)
We propose Weakly Coupled Deep Q-Networks (WCDQN), a new deep RL algorithm for weakly coupled MDPs, where the state and action spaces can be decomposed into nearly independent components except for a linking constraint on the action space.
Multi-Objective Reinforcement Learning for Sustainable Supply Chain Optimization
2023
Ibrahim El Shar, Haiyan Wang, Chetan Gupta
IEEE International Conference on Automation Science and Engineering (CASE) 2023
An approach to balance economic, environmental, and social objectives in supply chain optimization using multi-objective reinforcement learning.
Deep Reinforcement Learning toward Robust Multi-echelon Supply Chain Inventory Optimization
2022
Ibrahim El Shar, Wenhuan Sun, Haiyan Wang, Chetan Gupta
IEEE International Conference on Automation Science and Engineering (CASE) 2022
This paper is based on my work during my internship at Hitachi America, Ltd. R&D, focusing on inventory optimization in complex supply chains.
Spatial Dynamic Pricing for Shared-Resource Systems
2022
Ibrahim El Shar, Daniel Jiang
Work in Progress
We study the problem of spatial dynamic pricing for shared resources that circulate in a network. For the general network, we show that the optimal value function is concave and for a network composed of two locations, we show that the optimal policy enjoys certain monotonicity and bounded sensitivity properties. These results inform our novel heuristic and deep reinforcement learning algorithm.
Lookahead-Bounded Q-Learning
2020
Continuous inventory control with stochastic and non-stationary Markovian demand
2018
Walid W. Nasr, Ibrahim El Shar
European Journal of Operational Research, 2018
This paper addresses inventory control optimization under complex demand patterns with Markovian dynamics and non-stationarity.
Projects
Never-Ending Learning - Imitation Based Methods
2019-2020
Developed reinforcement learning and imitation learning methods for an endless grid-world environment with complex reward collection tasks.
- Proposed a novel greedy heuristic algorithm
- Compared performance against Deep Q-networks (DQN)
- Implemented imitation learning and Deep Q-learning from Demonstrations (DQfD)
IMDB Text Classification
2018-2019
Developed a novel sentence-based text representation for language classification problems, evaluated using the IMDB Movie Review Dataset.
- Created a new text representation approach
- Implemented feature extraction and classification algorithms
- Performed comparative analysis against baseline methods
Dynamic Airline Pricing
2018
Studied the problem of dynamic airline pricing with ticket return considerations, optimizing both initial ticket pricing and return value policies.
- Formulated as a dynamic programming problem
- Developed pricing strategies that maximize revenue
- Implemented simulation models to evaluate policy performance
Case Study For Gracious University Hospital
2018
Analyzed budget optimization for hospital inventory management, developing a simulation model for automated dispensing cabinets restocking policy.
- Implemented a Python simulation model using SymPy
- Designed an optimal (s, S) restocking policy
- Balanced patient needs with personnel requirements
Other work, interests, and hobbies
- Art: "I am seeking, I am stiving, I am in it with all my heart", Vincent Van Gogh. I am a self-taught artist that is driven by passion, seized by obsession and delighted by creation. Art has always been a huge passion of mine. I like to draw portraits and paint landscapes. Check out some of my work here.
- Proud to be a PITT Panther! I exercise daily and love going on walks and hikes.
- I am a big fan of chess, soccer and music.