[{"content":"PhD student under Pierre-Luc Bacon and Doina Precup. I split my time between research and building.\nResearch Working on questions like:\nDo we really need silicon-based computation? Can we build interfaces that adapt to users rather than the other way around? Can we train bigger neural networks in a distributed fashion? Tools like experiment_buddy aim to make AI researchers\u0026rsquo; workflows smoother.\nBook a chat: calendly.com/manuel-delverme/15min\nQuantified Self I track everything. 26 data sources spanning 13+ years: Spotify plays, GitHub commits, browser history, location data, messages, calendar events, finance transactions, health metrics.\nThe goal: identify causal factors in personal productivity through longitudinal self-experimentation. Most productivity advice relies on anecdote, survivorship bias, or small-N studies with poor ecological validity. With 13+ years of continuous measurement, I can run proper before/after analyses and control for confounds.\nWhat I\u0026rsquo;ve Found Some findings confirm conventional wisdom:\nMomentum matters (yesterday\u0026rsquo;s output predicts today\u0026rsquo;s) Morning focus correlates with higher output Some findings contradict it:\nMore meetings correlate with more commits, not fewer Late nights correlate with peak productivity (when you\u0026rsquo;re in flow) Some findings surprised me:\nSpotify skip rate \u0026gt;20% predicts a 55% drop in next-day commits Hands-up/eurodance music correlates with 6+ commits/day Media volume, not timing, predicts next-day crashes The Framework I think about life optimization as maximizing:\nV_total = ∫₀ᵀ V(t) dt Where V(t) = daily value, T = lifespan. Health extends T. Everything else optimizes V(t).\nDM me on Twitter or GitHub.\n","permalink":"https://manuel-delverme.github.io/about/","summary":"\u003cp\u003ePhD student under Pierre-Luc Bacon and Doina Precup. I split my time between research and building.\u003c/p\u003e\n\u003ch2 id=\"research\"\u003eResearch\u003c/h2\u003e\n\u003cp\u003eWorking on questions like:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eDo we really need silicon-based computation?\u003c/li\u003e\n\u003cli\u003eCan we build interfaces that adapt to users rather than the other way around?\u003c/li\u003e\n\u003cli\u003eCan we train bigger neural networks in a distributed fashion?\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eTools like \u003ca href=\"https://github.com/manuel-delverme/experiment_buddy\"\u003eexperiment_buddy\u003c/a\u003e aim to make AI researchers\u0026rsquo; workflows smoother.\u003c/p\u003e\n\u003cp\u003eBook a chat: \u003ca href=\"https://calendly.com/manuel-delverme/15min\"\u003ecalendly.com/manuel-delverme/15min\u003c/a\u003e\u003c/p\u003e\n\u003chr\u003e\n\u003ch2 id=\"quantified-self\"\u003eQuantified Self\u003c/h2\u003e\n\u003cp\u003eI track everything. 26 data sources spanning 13+ years: Spotify plays, GitHub commits, browser history, location data, messages, calendar events, finance transactions, health metrics.\u003c/p\u003e","title":"About"},{"content":"I\u0026rsquo;ve been tracking my digital life for over a decade. Spotify plays, GitHub commits, browser history, location data—everything feeds into a personal database I use to understand what actually drives my productivity.\nLast week, I stumbled onto something unexpected.\nI wasn\u0026rsquo;t looking for a correlation between music and code output. I was testing whether late-night screen time hurt next-day productivity. (It doesn\u0026rsquo;t—volume matters, not timing.) But in the process, I ran a query on Spotify skip rates.\nThe results stopped me cold.\nThe Dataset Before the finding, the context:\nSource Records Time Span Spotify plays 96,320 10+ years GitHub commits 6,186 11+ years Days with both 597 — Every Spotify play includes ms_played (how long I actually listened) and duration_ms (the full track length). A \u0026ldquo;skip\u0026rdquo; is any play under 30 seconds—the point where you\u0026rsquo;ve decided this isn\u0026rsquo;t what you want.\nI grouped days by skip rate and joined them to commit counts.\nThe Core Finding Skip Rate Days Avg Commits Delta \u0026lt;20% 578 5.45 baseline 20-39% 14 2.43 -55% 40%+ 5 2.40 -56% Read that again.\nOn days where I skipped more than 20% of my tracks, my code output dropped by 55%. Not 5%. Not 15%. More than half.\nThe sample size for high-skip days is small (19 days total), but the effect size is enormous. And it makes intuitive sense once you think about it.\nWhy Skipping Predicts Low Output Skipping isn\u0026rsquo;t just a music behavior. It\u0026rsquo;s a symptom of mental state.\nWhen you skip tracks, you\u0026rsquo;re:\nRejecting sustained attention — The song didn\u0026rsquo;t immediately satisfy, so you context-switch Seeking novelty — Dopamine-hunting rather than settling into a task Signaling restlessness — Your mind is bouncing, not focused Each skip is a micro-interrupt. You\u0026rsquo;re training your brain to reject anything that doesn\u0026rsquo;t provide instant gratification. That same brain then tries to write code—an activity that requires sustained focus across minutes or hours.\nIt doesn\u0026rsquo;t work.\nThe skip rate isn\u0026rsquo;t causing low productivity. It\u0026rsquo;s revealing it. By the time you\u0026rsquo;re skipping 20%+ of your music, your mind is already in a scattered state. The commits don\u0026rsquo;t happen because the focus was never there.\nThe Zero-Skip Artists If high skip rate indicates scattered attention, what does zero skip rate indicate?\nI pulled all artists where I\u0026rsquo;ve never skipped a single track (minimum 50 plays):\nArtist Plays Skip Rate Avg Sec Played Bass Modulators 50 0% 218 Code Black 62 0% 244 DJ Roxx 101 0% 203 Dancefloor Kingz 102 0% 197 Deep.Spirit 78 0% 239 East Clubbers 90 0% 221 Groove Coverage 50 0% 218 Horyzon 80 0% 265 Interphace 105 0% 238 Notice anything?\nThey\u0026rsquo;re all hands-up/eurodance artists. Every single one.\nThis isn\u0026rsquo;t coincidence. I\u0026rsquo;d previously found that hands-up music correlates with my highest commit days (6+ commits/day vs 3.2 baseline). Now I know part of why: the genre is predictable enough that it doesn\u0026rsquo;t trigger seeking behavior.\nHands-up tracks follow rigid formulas:\n4/4 beat, 140-150 BPM Verse → buildup → drop → breakdown → repeat Anthemic, repetitive vocals Your brain knows exactly what\u0026rsquo;s coming. There\u0026rsquo;s no novelty to seek. You settle in and work.\nContrast this with the high-skip artists:\nArtist Plays Skip Rate Fantasy Project 102 29.4% DJ Spooky 75 28.0% Dan Bull 154 25.3% Nightcore 394 16.0% These are more experimental, varied, or novelty-based. Each track is a gamble. Sometimes you love it; sometimes you don\u0026rsquo;t. That uncertainty creates seeking behavior—exactly what kills deep work.\nThe Time Signature Skip rate also varies by hour:\nTime Skip Rate State 3-4 AM 3.6% Deep flow 7-8 AM 7.9% Morning chaos 11-12 PM 7.4% Lunch scatter 9-11 PM 4.5-4.8% Evening flow The pattern maps directly to focus states:\nLate night (3-4 AM): If I\u0026rsquo;m still working at 3 AM, I\u0026rsquo;m in deep flow. Distractions have fallen away. I\u0026rsquo;m not seeking—I\u0026rsquo;m executing. Skip rate: 3.6%.\nMorning (7-8 AM): Warming up, checking messages, transitioning from sleep. Mind is fragmented. Skip rate: 7.9% (highest of the day).\nEvening (9-11 PM): Second wind. Fewer interruptions. Skip rate drops back to 4.5%.\nSkip rate isn\u0026rsquo;t just a daily metric—it\u0026rsquo;s a real-time flow indicator. If you notice yourself skipping tracks, you\u0026rsquo;re not in flow. You\u0026rsquo;re seeking.\nThe Intervention Data is useless without action. Here\u0026rsquo;s what I\u0026rsquo;m implementing:\n1. The 20% Circuit Breaker If I catch myself skipping more than 1 in 5 tracks during a work session:\nStop the music entirely.\nSilence is better than seeking. Continuing to hunt for the \u0026ldquo;right\u0026rdquo; song compounds the problem—each skip is another micro-interrupt, another dopamine hit from novelty.\nWhen the skip rate is high, the music isn\u0026rsquo;t helping. Remove it.\n2. The Zero-Skip Playlist I created a playlist exclusively from zero-skip artists. For deep work sessions, I start here. No decisions, no seeking, no skips.\nThe playlist is boring. That\u0026rsquo;s the point. Boring music fades into the background. Interesting music demands attention.\n3. Skip Rate as Diagnostic Before checking if I\u0026rsquo;m \u0026ldquo;being productive,\u0026rdquo; I now check: What\u0026rsquo;s my skip rate in the last hour?\n\u0026lt;10%: Green. Flow state likely. Keep going. 10-20%: Yellow. Attention is wandering. Consider a break or environment change. \u0026gt;20%: Red. Not in flow. Either take a real break or stop the music entirely. The skip rate tells me my mental state faster than any productivity app ever could. It\u0026rsquo;s a leading indicator, not a lagging one.\nThe Mechanism Why does this work? Here\u0026rsquo;s my hypothesis:\nSkipping trains your brain to reject sustained attention.\nEvery time you skip a track, you\u0026rsquo;re reinforcing a pattern:\nStimulus doesn\u0026rsquo;t immediately satisfy → reject it Seek new stimulus → get dopamine hit from novelty Repeat This is the same loop that makes social media addictive. Scroll, reject, scroll, reject. Each rejection feels productive (I\u0026rsquo;m finding the good stuff!) but is actually corrosive to focus.\nWhen you listen to a full track—even one you\u0026rsquo;re not in love with—you\u0026rsquo;re training the opposite pattern:\nStimulus doesn\u0026rsquo;t immediately satisfy → stay with it anyway Let it develop → find the payoff in sustained attention Build capacity for delayed gratification This is the same muscle you need for deep work. Code doesn\u0026rsquo;t provide instant gratification. The payoff comes after sustained focus. If you\u0026rsquo;ve spent your morning training your brain to reject anything that doesn\u0026rsquo;t immediately satisfy, you won\u0026rsquo;t have the capacity left for code.\nLimitations A few caveats:\nCorrelation isn\u0026rsquo;t causation. Skip rate doesn\u0026rsquo;t cause low productivity—it reveals an already-scattered mental state. But the intervention (stopping music when skip rate is high) might still help by removing a compounding factor.\nSmall sample for high-skip days. Only 19 days had skip rate \u0026gt;20%. The effect size is huge, but the sample is small. More data would strengthen confidence.\nPersonal data only. This is n=1. Your skip rate patterns might differ. But the mechanism (skipping as a symptom of scattered attention) should generalize.\nGenre preferences vary. Hands-up music works for me. It might be insufferable for you. The point isn\u0026rsquo;t the specific genre—it\u0026rsquo;s finding music that doesn\u0026rsquo;t trigger seeking behavior.\nConclusion I spent years optimizing the obvious productivity levers: sleep, exercise, calendar blocking, app blockers. None of them predicted my output with 55% accuracy.\nA metric I\u0026rsquo;d never considered—how often I skip Spotify tracks—turned out to be one of the strongest signals I have.\nSkip rate is a real-time flow indicator. High skipping reveals a scattered mind. Low skipping indicates settled attention. The correlation to code output is stronger than almost anything else in my dataset.\nThe intervention is simple:\nNotice when you\u0026rsquo;re skipping Stop the music Either take a real break or work in silence You can\u0026rsquo;t force flow. But you can stop the behaviors that prevent it.\nThis analysis is part of a larger project tracking 26 data sources across 13 years to understand what actually drives human performance. More findings coming soon.\n","permalink":"https://manuel-delverme.github.io/posts/skip-rate-signal/","summary":"\u003cp\u003eI\u0026rsquo;ve been tracking my digital life for over a decade. Spotify plays, GitHub commits, browser history, location data—everything feeds into a personal database I use to understand what actually drives my productivity.\u003c/p\u003e\n\u003cp\u003eLast week, I stumbled onto something unexpected.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eI wasn\u0026rsquo;t looking for a correlation between music and code output.\u003c/strong\u003e I was testing whether late-night screen time hurt next-day productivity. (It doesn\u0026rsquo;t—volume matters, not timing.) But in the process, I ran a query on Spotify skip rates.\u003c/p\u003e","title":"The Skip Rate Signal: How Your Spotify Habits Predict Tomorrow's Output"},{"content":"Decomposing a problem into smaller and more manageable pieces to be solved independently and then merging the solutions has been a common motif inspiring many fundamental advancements (Search, ordinary differential equations, Kepler etc.).\nIn this survey, I will present several approaches to divide the parameter estimation problem for neural networks by back-propagation into simpler sub-problems.\nNeural networks have recently had a strong impact in both our day to day life and in other fields, partially also due to this reason, the number of parameters of the modern models keeps increasing with no yet clear limits on performance, yet the hardware to train such networks won\u0026rsquo;t be able to keep up the pace of the software, the last 10 years have been supported by the move from CPUs to more specialized GPU hardware, recently dedicated hardware such as TPU or similar have managed to keep up with the demand but alas, there are limits to how many processing units we can fit in a small space, distributed algorithm have notoriously been the solution to such problem but we have yet to develop efficient optimization for dense problems like neural networks.\nThe problem not only lies in the real world limitations but also in the precision required to understand how to update a single parameter in a network of 175 billion other parameters.\nToday, we pay great attention to making the job of credit assignment to the network parameters as easy as possible by keeping the network gradients in check with techniques like residual networks, batch normalization, orthogonal initialization and others, this, like all heuristics adds some bias to the results.\nOverview In section 2 we will first see how we can reformulate the training of a neural network as a constrained optimization problem allowing us to turn global gradients into local ones, hereby allowing us to solve the global problem by solving many small problems. Section 3 will show how past methods tried to solve the constrained optimization problem, Section 4 will instead show how the field of optimal control encountered the same class of problems with a different name and we are going to see some of the most fundamental approaches.\nSection 5 will show how to efficiently solve large scale saddle point and finally, Section 6 will show how many current bio-inspired methods relate to the above framework.\nFeed-forward Neural Networks With the quest of subdivision we now look at a feed-forward neural network as a nested function in the form:\n$$\\hat{y}=\\phi(x_0; w)=f_{L-1}(\\dots f_i(\\dots f_0(x_0; w_0)\\dots;w_i)\\dots;w_L)$$\nAnd it\u0026rsquo;s associated loss function and optimization problem:\n$$\\underset{w}{\\operatorname{argmin}} \\ell(f_{L-1}(\\dots f_i(\\dots f_0(x_0; w_0)\\dots;w_i)\\dots;w_L), y)$$\nWhere $x_0$ is the given data-set and $y^{[j]}$ are the target values for each $x_0^{[j]}$ element, $\\ell$ is any kind of \u0026ldquo;loss function\u0026rdquo; that maps a set of network predictions $\\hat{y}^{[j]}$ to a scalar value.\nWe can specify this as a layer-wise sequence of functions $f_i$ applied to the output of the previous layer $h_i$:\n$$\\begin{cases} h_{1} = f_i(x_0; w_0)\\\\ \\vdots\\\\ h_{i+1} = f_i(h_i; w_i)\\\\ \\vdots\\\\ \\hat{y}=h_{L} = f_{L-1}(h_{L-1}, w_{L-1}) \\end{cases}$$\n$f_i(\\cdot; w_i)$ represents the input-output mapping (layer transfer function) of any type of feed-forward neural network layer such as fully connected or convolution layer including related non-linear activations.\nConstrained Formulation LeCun (1988) shows how to cleanly cast the back-propagation algorithm into an equivalent Lagrangian formulation.\nWe can now start decoupling the network layers by adding extra decision variables $h_i$ as targets for each layer. Each layer now has a target $h_i$ independent on the rest of the network, but as long as the constraints $h_{i+1} - f_i(h_i, w_i)$ are satisfied the two network dynamics are equivalent.\nThe new optimization problem is now:\n$$\\underset{(w, h)}{\\operatorname{argmin}} \\ell(h_L, y) \\quad \\text{s.t.} \\quad h_{i+1} = f_i(h_i; w_i)$$\nProblems of constrained optimization can be solved by the method of the Lagrangian multipliers where we define a Lagrangian function:\n$$L(x_0;w,h,\\lambda) = \\ell(h_L, y)+ \\lambda^T H(x_0; w, h)$$\nAnd seek to find $(w, h, \\lambda)$ s.t. $\\nabla L(x_0; w, h, \\lambda)=\\boldsymbol{0}$\nUnconstrained Optimization Algorithms While LeCun (1988) defines the problem statement and identifies sufficient conditions we are not provided with a way to solve the optimization problem. I will now show some proposed optimization methods to solve the constrained problem.\nMethod of the Auxiliary Coordinates Carreira-Perpinan \u0026amp; Wang (2012) introduces a mathematical formulation (meta-algorithm) named \u0026ldquo;MAC\u0026rdquo; to divide the training of a general neural network into sub-problems.\nThe search of parameters is done by using a penalty based method over $(w, h)$, solving a sequence of problems for $\\lambda_0 \\to \\infty$ by alternating the minimization over $w_i$ and $h_i$.\nGradient Descent Ascent Betti (2018) takes LeCun\u0026rsquo;s stationary conditions and defines a search approach in the $(w, x, \\lambda)$ space by gradient descent ascent (GDA).\nBlock-wise Training Gotmare introduces \u0026ldquo;BlockProp\u0026rdquo;, an algorithm that splits the unconstrained optimization problem into a constrained problem over blocks of $\\geq 1$ layers.\nDiscrete Time Optimal Control To better understand why this constrained optimization approaches, it might be useful to look back at other fields using the same tools to solve a different problem of long-term dependencies in control systems.\nThe problem of minimizing a final cost function (and optionally a layer-wise one too) is also found in optimal control where we look for a sequence of states $h_i$ and controls $w_i$ to minimize the costs $g_i$ and $\\ell(h_L, y)$ within the realizable dynamics $h_{i+1} = f_i(h_i; w_i)$.\nMain Approaches All trajectory methods can be described as either shooting methods or simultaneous collocation methods. Shooting methods use the system dynamics to explicitly forward the states from inputs $h_0$ to outputs $\\hat{y}$, while simultaneous collocation methods define a sequence of intermediate values $h_i$ and solve for the weights $w_i$.\nSingle Shooting These methods are analogous to classical back-propagation. Shooting methods and back-propagation have similar known issues:\nUninformative gradients: Shooting methods have local minima caused by the presence of narrow regions where the local information provided by the gradient is too little.\nDependency on parameter initialization: Both fields rely on good initial guesses to guarantee well-behaved optimization.\nMultiple Shooting Multiple shooting defines intermediate decision variables $h_i$ and imposes $h_{i+1} = f_i(h_i; w_i)$ constraints. Such methods lend themselves to parallelization.\nWith optimization using soft constraints, the iterates $(w, h)_k$ are allowed to exist outside the valid optimization space, allowing the optimization path to converge more robustly and with fewer iterations to the optimal solution.\nCollocation In collocation methods, we forget about the weights $w_i$ and look for a sequence of network activations $h_i$ that minimize the loss function.\nVariational Inequalities Up to now, we have seen different perspectives of the same problem, solving these problems at the scale of modern datasets would prove challenging. We would like to have a first-order algorithm with good enough convergence speed. The field of Variational Inequalities can provide such answers.\nExtragradient (EG) If we look inside the function we want to optimize we see that it has the form of a two-player $(\\theta, \\lambda)$ game. Gidel shows that in even the simplest bi-linear games Gradient Descent Ascent (GDA) fails to converge, this is caused by rotations around the fixpoint.\nKorpelevich (1976) dampens circling behaviour by performing an extrapolation step (look ahead) for both players and applying the extrapolated gradients instead of the original ones.\nExtrapolation from the Past Gidel introduces an \u0026ldquo;extrapolation from the past\u0026rdquo;, allowing us to avoid the computation cost of evaluating two gradients at every iterate.\nBrain-inspired Back-Propagation This section reviews alternative optimization methods. These works depart from the optimization view and often also from accurate optimization objectives (which is a proxy for the real generalization objective anyway) in favour of other heuristic objectives.\nNone of the methods try to exactly mimic the brain but rather draw inspiration from nature. One of the major points is that to reproduce accurately back-propagation, each neuron should store both forward (primal) and backward (tangent) gradient and their values which is not empirically observed in the brain.\nTarget Propagation Krogh (1989) solves the Lagrangian with L1 distance and a fixed constant by gradient descent. They also show how to impose per hidden constraints to control the learned hidden representations.\nDifference Target Propagation Global gradients require knowledge of the full network and this is not biologically plausible. Lee (2015) side steps the issue relying on target values and approximate inverse dynamics. This method is interesting because it encodes internal models for forward and backward dynamics by auto-encoders.\nNeural Gradient Representation by Activity Differences Lillicrap (2020) asks whether the brain is in fact approximating backpropagation. They introduce an NGRAD Hypothesis which suggests the human cortex uses an NGRAD based method to approximate gradient descent because of the impossibility of estimating global gradients.\nConclusion One of the controversial points of backpropagation in neural networks is the lack of local computation, yet it works so well there is no reason to move away from it.\nAs we scale the size of neural networks we are starting to question the current optimization methods. Problems like long term credit assignment has been part of the field of discrete-time optimal control and solved with some degree of success, so it seems worth a try to stand on the shoulders of those giants.\nThere are still open questions such as how to use stochastic optimization methods, which Gotmare shows some degree of success by ignoring the problem and sampling the constraints as well but this approach has not been shown to scale to modern datasets.\n","permalink":"https://manuel-delverme.github.io/posts/constrained-neural-networks/","summary":"\u003cp\u003eDecomposing a problem into smaller and more manageable pieces to be solved independently and then merging the solutions has been a common motif inspiring many fundamental advancements (Search, ordinary differential equations, Kepler etc.).\u003c/p\u003e\n\u003cp\u003eIn this survey, I will present several approaches to divide the parameter estimation problem for neural networks by back-propagation into simpler sub-problems.\u003c/p\u003e\n\u003cp\u003eNeural networks have recently had a strong impact in both our day to day life and in other fields, partially also due to this reason, the number of parameters of the modern models keeps increasing with no yet clear limits on performance, yet the hardware to train such networks won\u0026rsquo;t be able to keep up the pace of the software, the last 10 years have been supported by the move from CPUs to more specialized GPU hardware, recently dedicated hardware such as TPU or similar have managed to keep up with the demand but alas, there are limits to how many processing units we can fit in a small space, distributed algorithm have notoriously been the solution to such problem but we have yet to develop efficient optimization for dense problems like neural networks.\u003c/p\u003e","title":"Divide et Impera and Neural Networks"}]