Rebalance improvement.

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Rebalance improvement.

Susant Palai
Hi,
    Recently, we have pushed some performance improvements for Rebalance Crawl which used to consume a significant amount of time, out of the entire rebalance process.


The patch [1] is recently merged in upstream and may land as an experimental feature in the upcoming upstream release.

The improvement currently works only for pure-distribute Volume. (which can be expanded).


Things to look forward to in future :
 - Parallel Crawl in Rebalance
 - Global Layout

Once these improvements are in place, we would be able to reduce the overall rebalance time by a significant time.

Would request our community to try out the feature and give us feedback.

More information regarding the same will follow.


Thanks & Regards,
Susant Palai



_______________________________________________

Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968




Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: Rebalance improvement.

Susant Palai
Centos Users can add the following repo and install the build from the master branch to try out the feature. [Testing purpose only, not ready for consumption in production env.]

[gluster-nightly-master] baseurl=http://artifacts.ci.centos.org/gluster/nightly/master/7/x86_64/ gpgcheck=0 keepalive=1 enabled=1 repo_gpgcheck = 0 name=Gluster Nightly builds (master branch)

A summary of perf numbers from our test lab :

DirSize - 1Million Old New %diff Depth - 100 (Run 1) 353 74 +377% Depth - 100 (Run 2) 348 72 +377~% Depth - 50 246 122 +100% Depth - 3 174 114 +52%

Susant


On Mon, Aug 3, 2020 at 11:16 AM Susant Palai <[hidden email]> wrote:
Hi,
    Recently, we have pushed some performance improvements for Rebalance Crawl which used to consume a significant amount of time, out of the entire rebalance process.


The patch [1] is recently merged in upstream and may land as an experimental feature in the upcoming upstream release.

The improvement currently works only for pure-distribute Volume. (which can be expanded).


Things to look forward to in future :
 - Parallel Crawl in Rebalance
 - Global Layout

Once these improvements are in place, we would be able to reduce the overall rebalance time by a significant time.

Would request our community to try out the feature and give us feedback.

More information regarding the same will follow.


Thanks & Regards,
Susant Palai



_______________________________________________

Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968




Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: [Gluster-users] Rebalance improvement.

Aravinda VK-2
Interesting numbers. Thanks for the effort.

What is the unit of old/new numbers? seconds? 

On 03-Aug-2020, at 12:47 PM, Susant Palai <[hidden email]> wrote:

Centos Users can add the following repo and install the build from the master branch to try out the feature. [Testing purpose only, not ready for consumption in production env.]

[gluster-nightly-master] baseurl=http://artifacts.ci.centos.org/gluster/nightly/master/7/x86_64/ gpgcheck=0 keepalive=1 enabled=1 repo_gpgcheck = 0 name=Gluster Nightly builds (master branch)

A summary of perf numbers from our test lab :

DirSize - 1Million Old New %diff Depth - 100 (Run 1) 353 74 +377% Depth - 100 (Run 2) 348 72 +377~% Depth - 50 246 122 +100% Depth - 3 174 114 +52%

Susant


On Mon, Aug 3, 2020 at 11:16 AM Susant Palai <[hidden email]> wrote:
Hi,
    Recently, we have pushed some performance improvements for Rebalance Crawl which used to consume a significant amount of time, out of the entire rebalance process.


The patch [1] is recently merged in upstream and may land as an experimental feature in the upcoming upstream release.

The improvement currently works only for pure-distribute Volume. (which can be expanded).


Things to look forward to in future :
 - Parallel Crawl in Rebalance
 - Global Layout

Once these improvements are in place, we would be able to reduce the overall rebalance time by a significant time.

Would request our community to try out the feature and give us feedback.

More information regarding the same will follow.


Thanks & Regards,
Susant Palai


________



Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968

Gluster-users mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-users

Aravinda Vishwanathapura




_______________________________________________

Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968




Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: [Gluster-users] Rebalance improvement.

Susant Palai


On 03-Aug-2020, at 13:58, Aravinda VK <[hidden email]> wrote:

Interesting numbers. Thanks for the effort.

What is the unit of old/new numbers? seconds? 

Minutes. 


On 03-Aug-2020, at 12:47 PM, Susant Palai <[hidden email]> wrote:

Centos Users can add the following repo and install the build from the master branch to try out the feature. [Testing purpose only, not ready for consumption in production env.]

[gluster-nightly-master] baseurl=http://artifacts.ci.centos.org/gluster/nightly/master/7/x86_64/ gpgcheck=0 keepalive=1 enabled=1 repo_gpgcheck = 0 name=Gluster Nightly builds (master branch)

A summary of perf numbers from our test lab :

DirSize - 1Million Old New %diff Depth - 100 (Run 1) 353 74 +377% Depth - 100 (Run 2) 348 72 +377~% Depth - 50 246 122 +100% Depth - 3 174 114 +52%

Susant


On Mon, Aug 3, 2020 at 11:16 AM Susant Palai <[hidden email]> wrote:
Hi,
    Recently, we have pushed some performance improvements for Rebalance Crawl which used to consume a significant amount of time, out of the entire rebalance process.


The patch [1] is recently merged in upstream and may land as an experimental feature in the upcoming upstream release.

The improvement currently works only for pure-distribute Volume. (which can be expanded).


Things to look forward to in future :
 - Parallel Crawl in Rebalance
 - Global Layout

Once these improvements are in place, we would be able to reduce the overall rebalance time by a significant time.

Would request our community to try out the feature and give us feedback.

More information regarding the same will follow.


Thanks & Regards,
Susant Palai


________



Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968

Gluster-users mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-users

Aravinda Vishwanathapura





_______________________________________________

Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968




Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: [Gluster-users] Rebalance improvement.

Sankarshan Mukhopadhyay
In reply to this post by Susant Palai
On Mon, 3 Aug 2020 at 12:47, Susant Palai <[hidden email]> wrote:
>
> Centos Users can add the following repo and install the build from the master branch to try out the feature. [Testing purpose only, not ready for consumption in production env.]
>
> [gluster-nightly-master] baseurl=http://artifacts.ci.centos.org/gluster/nightly/master/7/x86_64/ gpgcheck=0 keepalive=1 enabled=1 repo_gpgcheck = 0 name=Gluster Nightly builds (master branch)
>
> A summary of perf numbers from our test lab :
>

Are these numbers impacted by sizing of the machine instance/hardware?
What is the configuration on which these numbers were recorded?

> DirSize - 1Million Old New %diff Depth - 100 (Run 1) 353 74 +377% Depth - 100 (Run 2) 348 72 +377~% Depth - 50 246 122 +100% Depth - 3 174 114 +52%
>
> Susant

--
sankarshan mukhopadhyay
<https://about.me/sankarshan.mukhopadhyay>
_______________________________________________

Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968




Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel

Reply | Threaded
Open this post in threaded view
|

Re: [Gluster-users] Rebalance improvement.

Susant Palai


On Mon, Aug 3, 2020 at 2:53 PM sankarshan <[hidden email]> wrote:
On Mon, 3 Aug 2020 at 12:47, Susant Palai <[hidden email]> wrote:
>
> Centos Users can add the following repo and install the build from the master branch to try out the feature. [Testing purpose only, not ready for consumption in production env.]
>
> [gluster-nightly-master] baseurl=http://artifacts.ci.centos.org/gluster/nightly/master/7/x86_64/ gpgcheck=0 keepalive=1 enabled=1 repo_gpgcheck = 0 name=Gluster Nightly builds (master branch)
>
> A summary of perf numbers from our test lab :
>

Are these numbers impacted by sizing of the machine instance/hardware?
What is the configuration on which these numbers were recorded?

There were 4 bricks(one brick per node) to begin with. Post creation of directories, 2 more bricks(from two more nodes) were added.

Disks - HDD
Network - 10Gbps ethernet link
cores -  24

Numbers will definitely vary according to disk, network configuration.
 

> DirSize - 1Million Old New %diff Depth - 100 (Run 1) 353 74 +377% Depth - 100 (Run 2) 348 72 +377~% Depth - 50 246 122 +100% Depth - 3 174 114 +52%
>
> Susant

--
sankarshan mukhopadhyay
<https://about.me/sankarshan.mukhopadhyay>
_______________________________________________

Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968




Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel


_______________________________________________

Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968




Gluster-devel mailing list
[hidden email]
https://lists.gluster.org/mailman/listinfo/gluster-devel