Measure, report and develop against estimated CO2 emissions #2

New Issue

Open

opened 2023-06-28 08:57:13 +00:00 by yarmo · 1 comment

yarmo commented

2023-06-28 08:57:13 +00:00

Owner

The overall concept would likely be to:

measure data usage and traffic by the various Keyoxide subprojects
estimate CO2 emissions
report and track those numbers
make (development) decisions with the goal to minimize the estimated emissions or stay within a budget

Overall, I think the idea is interesting and something that is ignored by the larger dev community. If we could implement a clear strategy to integrate those estimates, we could hopefully inspire other projects to follow a similar path.

Since estimations of CO2 emissions are already by nature an inaccurate science, thoughtful considerations must be made lest the method becomes entirely pseudo-scientific and hurts the developer experience for no actual benefit.

In a discussion this morning on the Keyoxide General Discussion Matrix channel, an idea was proposed by @Ryuno-Ki to integrate CO2 emission estimations in the development process of the Keyoxide project. The overall concept would likely be to: - measure data usage and traffic by the various Keyoxide subprojects - estimate CO2 emissions - report and track those numbers - make (development) decisions with the goal to minimize the estimated emissions or stay within a budget Overall, I think the idea is interesting and something that is ignored by the larger dev community. If we could implement a clear strategy to integrate those estimates, we could hopefully inspire other projects to follow a similar path. Since estimations of CO2 emissions are already by nature an inaccurate science, thoughtful considerations must be made lest the method becomes entirely pseudo-scientific and hurts the developer experience for no actual benefit.

Ryuno-Ki commented

2023-06-28 13:06:28 +00:00

Thanks for the mention. I want to elaborate a bit more.

I've brought up a project I'm keeping an eye on since some time: co2.js.

Preface

I want to start with what led me to bringing this subject up before I sketch on how I would imagine this to be handled. Don't feel compelled to do it as I say but pick what you like.

Precondition

Our world is burning. We emit more CO2 etc than the planet can handle. IT has to play its role in this.

Motivation

Reading The Experiences of People Running Mastodon Servers Suggest a Bumpy Road Ahead
brought this subject to the top of my mind again.

I agree with you that sustainability is a subject that goes largely unnoticed. That being said, there are conferences like Bits & Bäume that focus on these areas. Buzzword Green computing.

Combine that with Web development's lost decade on The Changelog and you realise that we are doing a bad job.

I can imagine that this is in part caused by the Eternal September effect as that more and more people break into tech (which is great!) than are taught the craftmanship. In fact, I feel like especially JavaScript is picking up on techniques that other languages have had for years. So do other languages adopt Functional Programming concepts. Indeed, we have lost time.

So I want to take this project to educate in part. I'm speaking with a decade of experience in the industry. A part of it even in DevOps.

Implementation

The target audience in my mind are operators. Those folks that feel a Devotion to Duty :)

In second order developers should look into this.

For the services I want to run myself I want to implement Observability. This grants me insights into how my software is doing. After doing some research I decided to go with Telegraf for my on-premise solution. It features lots of integrations to give you a glimpse of what is possible.
I'm pairing this with Grafana for dashboarding which also comes with plenty of data sources.

Relevance for Keyoxide

Keyoxide is talking to different other APIs. As such I think it makes sense to have some metrics on how those other services behave. My services are behind Nginx as reverse proxy. This web server can log request time for requests declared as upstream.

If such a service takes long time to respond, future requests should not be made and the user be shown a warning that the service is currently degraded. Best approach for that person is to try again later.

But network traffic is not the only source of emissions Keyoxide (or other projects for that matter) are causing. Algorithms is the main thing that is in control.

The CO2 project above suggested to define a budget. The JavaScript world knows about performance budgets that work in a similar fashion. Even if no one can be defined, the trend is already valuable.

Speaking of numbers, I'm not sure whether the values are accurate. In fact I would assume that the datasets have some error baked in. But I would expect that the order of magnitude is about right. This alone should be valuable (if you plot, the order of magnitude for result in a logplot - that is, the axis are logarithms.

In order to be applicable to a rust crate, I think the code would need to be ported. So at the current step, take at as an opportunity to have a conversation on it. That's all I'm asking for right now.

If I get around to add it as a datasource and you are interested, let me know :)

Thanks for the mention. I want to elaborate a bit more. I've brought up a project I'm keeping an eye on since some time: [co2.js](https://github.com/thegreenwebfoundation/co2.js). # Preface I want to start with what led me to bringing this subject up before I sketch on how I would imagine this to be handled. Don't feel compelled to do it as I say but pick what you like. ## Precondition Our world is burning. We emit more CO2 etc than the planet can handle. IT has to play its role in this. ## Motivation Reading [The Experiences of People Running Mastodon Servers Suggest a Bumpy Road Ahead ](https://cheapskatesguide.org/articles/mastodon-servers.html) brought this subject to the top of my mind again. I agree with you that sustainability is a subject that goes largely unnoticed. That being said, there are conferences like [Bits & Bäume](https://bits-und-baeume.org/en/) that focus on these areas. Buzzword [Green computing](https://en.wikipedia.org/wiki/Green_computing). Combine that with [Web development's lost decade](https://changelog.com/jsparty/263) on The Changelog and you realise that we are doing a bad job. I can imagine that this is in part caused by the Eternal September effect as that more and more people break into tech (which is great!) than are taught the craftmanship. In fact, I feel like especially JavaScript is picking up on techniques that other languages have had for years. So do other languages adopt Functional Programming concepts. Indeed, we have lost time. So I want to take this project to educate in part. I'm speaking with a decade of experience in the industry. A part of it even in DevOps. # Implementation The target audience in my mind are operators. Those folks that feel a [Devotion to Duty](https://xkcd.com/705/) :) In second order developers should look into this. For the services I want to run myself I want to implement [Observability](https://en.wikipedia.org/wiki/Observability_(software)). This grants me insights into how my software is doing. After doing some research I decided to go with Telegraf for my on-premise solution. It features [lots of integrations](https://www.influxdata.com/products/integrations/) to give you a glimpse of what is possible. I'm pairing this with Grafana for dashboarding which also comes with [plenty of data sources](https://grafana.com/grafana/plugins/?type=datasource). ## Relevance for Keyoxide Keyoxide is talking to different other APIs. As such I think it makes sense to have some metrics on how those other services behave. My services are behind Nginx as reverse proxy. This web server can [log request time](https://nginx.org/en/docs/http/ngx_http_core_module.html#var_request_time) for requests declared as upstream. If such a service takes long time to respond, future requests should not be made and the user be shown a warning that the service is currently degraded. Best approach for that person is to try again later. But network traffic is not the only source of emissions Keyoxide (or other projects for that matter) are causing. Algorithms is the main thing that is in control. The CO2 project above suggested to define a budget. The JavaScript world knows about [performance budgets](https://web.dev/performance-budgets-101/) that work in a similar fashion. Even if no one can be defined, the trend is already valuable. Speaking of numbers, I'm not sure whether the values are accurate. In fact I would assume that the [datasets](https://www.thegreenwebfoundation.org/green-web-datasets/) have some error baked in. But I would expect that the order of magnitude is about right. This alone should be valuable (if you plot, the order of magnitude for result in a [logplot](https://en.wikipedia.org/wiki/Log%E2%80%93log_plot) - that is, the axis are logarithms. In order to be applicable to a rust crate, I think the code would need to be ported. So at the current step, take at as an opportunity to have a conversation on it. That's all I'm asking for right now. If I get around to add it as a datasource and you are interested, let me know :)

No Label

No Milestone

No project

No Assignees

2 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: keyoxide/project#2

There is no content yet.